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Compendium of Key Studies of the No Child Left Behind Act 



During the coming months, the Congress and Administration are iikeiy to make important decisions about 
the federai roie in eiementary and secondary education. The No Chiid Left Behind Act (NCLB), which 
amended the Eiementary and Secondary Education Act (ESEA) of 1965, is up for reauthorization. The Act 
addresses a iarge portion of the main federal programs affecting K-12 schoois. 

As part of a iarger project to rethink the federai roie in eiementary and secondary education, CEP has 
prepared this compendium, which summarizes the findings of major studies of the impiementation and 
effects of NCLB conducted by various organizations and agencies. The summaries are very brief — most of 
them one page per study — and have been categorized by 

Both this compendium and CEP's broader federai roie project are intended to heip poiicymakers make 
decisions informed by evidence from research. Because the summaries are very condensed, we see this 
compendium as a starting point to famiiiarize poiicymakers and other with the range of research avaiiabie on 
NCLB. We encourage readers to use the Web iinks at the bottom of each summary to expiore the fuil 
reports in more detaii. 

This edition of the compendium focuses primariiy on studies of Titie I of ESEA as amended by NCLB, the 
federal program to improve education for iow-achieving chiidren in iow-income areas. New studies and other 
topics may be added iater. 

Criteria for Inciuding Studies 

We inciuded studies that met the foliowing criteria: 



• Were pubiished in 2005 or iater, after states, districts, and schoois had become famiiiar with the 
iaw's requirements and impiementation was weii underway 

• Had a nationai or regionai scope or inciuded evidence from muitiple states and/or schooi districts, 
rather than being focused on a singie state or one or two districts 

• Were conducted by a research organization, government agency, university, nationai organization 
with a research division, or schoiars with expertise in NCLB issues 

• Were based on data coilected through weli-estabiished research methods rather than being 
primariiy opinion pieces 

• Focused on a significant poiicy issue or outcome of NCLB 

• Couid be accessed free of charge, at ieast in summary form, on the Web 

Appiying some of these criteria necessariiy invoived making judgment caiis — deciding, for example, what 
constitutes a significant poiicy issue or where to draw the line between pure opinion and informed 
interpretation of research. We tried to make these judgments in good faith and without regard to our own 
views about the findings of the studies. 

Process for Developing the Summaries 

A CEP senior consuitant and two research interns reviewed the findings of the selected studies and 
deveioped summaries of each. We based our summaries on the executive summaries, conciusions, or other 
summary sections of the originai study reports. For this reason, much of the writing credit belongs to the 
original authors of the studies. To keep each summary to a page or two, however, we used a broad-brush 
approach and often further condensed the findings from those in the originai reports. 

Two experts who have closeiy tracked NCLB research since the iaw's inception reviewed the iist of studies 
and the content of the summaries and suggested additions and revisions. 




Organization of the Compendium 



The compendium is divided into nine sections and an index: 



• Accountabiiitv and reiated issues 

• Achievement and reiated issues 

• Curricuium and instruction 

• Funding, costs, and capacity 

• Generai impiementation 

• Reading First 

• School improvement and restructuring 

• Supplemental educational services and school choice 

• Teacher gualitv 

• Index of studies 



Within each section, studies are listed in alphabetical order by their main sponsoring organization. Where 
there are multiple studies by the same organization, they are listed from the most recent to the oldest. For 
comprehensive studies that address several topics, relevant findings may be mentioned in more than one 
category. 

The Index of Studies includes a complete alphabetical list of studies summarized in the compendium. The 
list shows the sponsoring organization, date, author, and title of the study and which section they can be 
found in. 

Common Abbreviations Used in the Summaries 

AYR — Adequate yearly progress 
ED — U. S. Department of Education 
ELLs — English language learners 

ESEA — Elementary and Secondary Education Act of 1965 

MOUSSE — High objective uniform state standard of evaluation (for teacher qualifications) 

NAEP — National Assessment of Educational Progress 

NCLB — No Child Left Behind Act 

SES — Supplemental educational services 
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Davaney-Graham, CEP interns. Diane Stark Rentner, CEP's director of national programs, provided advice, 
direction, and editing. CEP would like to thank Wayne Riddle and Joel Packer for reviewing the list of studies 
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American Federation of Teachers, 2008 

Sizing up state standards 2008* 



Focus 

J udges the clarity and specificity of states' academic content standards in light of NCLB 
accountability requirements. 

Methodology 

Examined state content standards in four core content areas (English, math, science and 
social studies) posted on state Web sites as of October 2007 for all 50 states and D.C. 
Analyzed the standards using a set of criteria intended to determine whether standards 
contained enough information about what students should learn to provide the basis for 
coherent curricula and assessments. 

Major Findings 

• Lack of strong standards in most states. Only one state, Virginia, met the AFT 

criteria for strong standards in all levels and subjects. While some states have a lot 
of work ahead of them to improve their standards, others have to focus on only a 
few grades in one subject area. 

• Better standards in math and science. Since 1995, states have done a better job 
with their standards in math and science than in English and social studies. Twenty- 
four states have strong math standards, and 22 have strong science standards. Only 
8 states have strong English standards at all levels, and only 2 have strong social 
studies standards at all levels. 

• Variation by level. For most subjects, middle school standards are the strongest, 
while high school standards are the weakest. The weaknesses at the high school 
level, in many cases, are due to the high school standards being clustered (e.g., one 
set of standards for grades 9-12) instead of being grade- or course-specific. 

• Reasons for weaker standards. Standards failed to meet AFT criteria for three 
main reasons: They were repeated, clustered, or had missing or vague content. All 
three of these problems have the same consequences. Teachers do not have a 
common understanding of what students should have learned in the previous grade, 
what they are expected to master in the current grade, or what they are preparing 
them to learn in the following grade. 

• Early grade standards. Too many states have clustered K-2 standards or have 
chosen not to write them at all. Nine states have clustered or no standards for K-2 in 
literacy and numeracy. Specific, coherent, grade-by-grade standards at the early 
grades are essential to building students' background knowledge and vocabulary. 

Where to Obtain 

http://www.aft.ora/pubs-reports/downloads/teachers/standards2008.pdf 



For additional information on this topic, see the summary of the 2006 AFT report. Smart Testing: Let's Get It 
Right. 
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American Federation of Teachers, 2006 

Smart testing: Let's get it right. How assessment-savvy have states become since NCLB?* 

Focus 

Examines whether states' academic content standards are clear and specific for each 
subject and grade tested and whether state assessments in reading, math, and science are 
aligned with strong standards in light of NCLB's emphasis on test-based accountability. 

Methodology 

Reviewed 861 content standards documents (357 in reading, 357 in math, and 147 in 
science) posted on the state Web sites for all states and D.C in 2006. Also reviewed 
information about 833 state tests (357 in reading, 357 in math, and 119 in science). 

Major Findings 

• Alignment of standards and tests. Seventy-four percent of the content standards 
across states met AFT's criteria for strong standards, but only 52% of states' tests 
were aligned to strong standards. States had done a better job of developing content 
standards than using them to drive assessment. Testing unaligned to strong 
standards was driving many accountability systems in 2006. As a result, the testing 
systems in many states were not yet "smart" enough to bear the weight of the 
accountability functions they were being asked to serve. 

• Specificity of standards. Overall, content standards were more specific in 2006 
than when AFT reviewed standards in 2001. This was particularly notable given that 
states had to develop 17 different sets of content standards after enactment of 
NCLB, versus the 9 required in 2001. 

• Weaknesses. Some states did not post any information on their Web sites about 
which standards were being assessed by the tests they administered. States 
continued to struggle with articulating strong reading standards, including helpful 
descriptions of what students need to know at each grade level. States also had not 
articulated clearly enough what high school students should be learning in reading, 
math, and science. Nearly half of the testing (48%) was based on weak standards 
and/or unaligned tests. 

Where to Obtain 

http://www.aft.orq/presscenter/ releases/2006/smarttestinq/Testinqbrief.pdf 



For additional recent information on this topic, see the summary of the 2008 AFT report. Sizing Up State 
Standards. 
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Arizona State University, Education Policy Research Unit, 2006 

The accuracy and effectiveness of adequate yearly progress, NCLB's school evaluation 

system 



Focus 

Considers how well adequate yearly progress is working as the key element of the NCLB 

accountability system. 

Methodology 

Analyzed evidence from a variety of studies about major aspects of AYP. 

Major Findings 

• Lack of evidence for AYP requirements. I n its 2006 form, AYP as an indicator of 
academic performance was not supported by reliable evidence. Nor was there any 
evidence at the time of this study that AYP demands, by themselves, will adequately 
provide schools serving children in poverty with the facilities, learning resources, 
qualified staff, or community support services needed to improve achievement. 

• 100% proficient unrealistic. Test score gains that may be attributable to the AYP 
process are modest and insufficient to achieve the goal of 100% proficiency by 2014. 

• Insufficient funding. Whether conceived as implementation costs or remedial 
costs, NCLB has been significantly underfunded in a way that disproportionately 
penalizes schools attended by the neediest children. 

• Curriculum narrowing. The coupling of high-stakes consequences with test scores 
increasingly appears to narrow curriculum in the opinion of state and local educators. 

• Flexibility measures not a solution. Growth models and other "flexibility" 
measures do not have sufficient power to resolve the underlying problems of the AYP 
system. For example, all students must still meet the same standards by 2014 under 
a growth model. If the minimum size for disaggregated groups is reduced, the sum 
effect would be to make an already unworkable system less workable. 

• Recommendation to suspend AYP. The report recommends that AYP sanctions be 
suspended until the premises underlying them can be either confirmed or refuted by 
solid, scientific research and unintended, negative consequences can be avoided. 

Where to Obtain 

http://epsl.asu.edu/epru/documents/EPSL-0609-212-EPRU.pdf 
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Arizona State University, Education Policy Research Unit, 2005 

High-stakes testing and student achievement: Problems for the No Child Left Behind Act 

Focus 

Investigates whether the pressure of high-stakes testing has influenced students' academic 
performance 

Methodology 

To measure the impact of high-stakes testing pressure on achievement and to account for 
the differences in testing pressure among the states, the researchers created a Pressure 
Rating Index (PRI), which sought to capture the amount of pressure or "threat" associated 
with performance on a particular test. The index was developed by reviewing state legal 
requirements, interviewing state officials, and consulting media sources. The degree of 
pressure associated with specific state tests was determined by groups of graduate- level 
education students. The researchers analyzed correlations between the PRI and NAEP 
results from 1990 to 2003 in 25 states and used the PRI to replicate previous research. 

Major Findings 

• No important influence. Pressure created by high-stakes testing has had almost 
no important influence on student academic performance. 

• Greater test pressure in states with more minority students. States with 
greater proportions of minority students have implemented accountability systems 
that exert greater pressure. This finding suggests that problems linked to high- 
stakes testing will disproportionately affect minority students. 

• Greater test pressure and dropout rates. High-stakes testing pressure is 
negatively associated with the likelihood that 8*^^ and graders will still be in 
school in 12‘^ grade. Increased testing pressure appears to be related to larger 
numbers of students being held back or dropping out of school. 

• No link between pressure and NAEP reading scores. Increased testing pressure 
produced no gains in NAEP reading scores for students in grades 4 or 8. 

• Weak link in grade 4 math. After 1996, increased testing pressure was weakly 
linked to later gains in NAEP math achievement at grade 4 for all ethnic subgroups. 

In grade 8 math, however, the study found no evidence that earlier pressure 
increases produced later achievement gains. The study authors noted that since 
math in the primary grades is far more standardized across the country than the 
math curriculum in middle school, drilling students and teaching to the test, as well 
as increased test pressure, could have played a role in these achievement gains. 

Where to Obtain 

http://epsl.asu.edu/epru/documents/EPSL-0509-105-EPRU.pdf 
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The Aspen Institute, Commission on No Child Left Behind, 2007 

Children with disabilities and LEP students: Their impact on the AYP determinations of 

schools 



Focus 

Examines the impact of minimum subgroup size requirements in five states on 
determinations of adequate yearly progress for students with disabilities and English 
language learners. 

Methodology 

Collected student achievement data from school year 2004-05 for all schools in California, 
Florida, Michigan, Georgia, and Pennsylvania. 

Major Findings 

• Impact of minimum subgroup size. State-set requirements for minimum n size— 
the number of students that must be in a subgroup for that subgroup to count for 
AYP purposes— can have a considerable impact on the AYP status of a school. Since 
2002, the general trend has been for states to increase their n sizes. Consequently, 
fewer schools are being held accountable for the performance of specific subgroups 
because these subgroups fall below their states' minimum n size. 

• Schools with SWD and ELL subgroups too small to count. The percentages of 
schools in the five states that did not have to report AYP determinations for the 
subgroup of students with disabilities ranged from 41% in Michigan to 92% in 
California. The percentages that did not have to report AYP determinations for the 
English language learner subgroup ranged from to 56% in California to 99% in 
Pennsylvania. 

• Students with disabilities and ELLs not sole reason for missing AYP. Findings 
from the five states studied raise questions about claims that many schools do not 
make AYP solely based on performance of students with disabilities and ELLs. Even 
when these subgroups fell short of AYP targets, they were very often not the sole 
reason why a school failed to make AYP. 

• State data. Specific data from the five states are shown in the two tables below: 



I mpact of Students with Disabilities (SWDs) on Adequate Yearly Progress 



State & n size* 


% of all schools 
reporting AYP for 
subgroup 


% of all schools 
that missed AYP 
in subgroup 


% of schools not 
making AYP that 
missed solely 
due to subgroup 


% of all tested SWDs 
enrolled in schools 
reporting AYP for 
subgroup 


CA - 100, or 50 
and 15% 


9% 


4% 


1% 


28% 


FL - 30 and 15% 


58% 


22% 


2% 


83% 


GA - 40 or 10% 


53% 


10% 


38% 


80% 


Ml - 30 


60% 


3% 


12% 


70% 


> 

o 

-1- 


11% 


6% 


19% 


41% 



Percentages in n size coiumn refer to percentage of totai schooi enroiiment. 
^State has speciai provisions for caicuiating AYP in schoois with n size beiow 40. 
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I mpact of English Language Learners on Adequate Yearly Progress 



state & n size* 


% of all schools 
reporting AYP for 
subgroup 


% of all schools 
that missed AYP 
in subgroup 


% of schools not 
making AYP that 
missed solely 
due to subgroup 


% of all tested ELLs 
enrolled in schools 
reporting AYP for 
subgroup 


CA - 100, or 50 
and 15% 


44% 


22% 


12% 


87% 


FL - 30 and 15% 


23% 


14% 


1% 


80% 


GA - 40 or 10% 


10% 


1% 


2% 


67% 


Ml - 30 


9% 


<1% 


2% 


45% 


> 

O 


1% 


<1% 


0% 


20% 



*Percentages in n size column refer to percentage of total school enrollment. 
^State has special provisions for calculating AYP in schools with n size below 40. 



Where to Obtain 

http://www.aspeninstitute.org/atf/cf/%7BDEB6F227-659B-4EC8-8F84- 
8DF23CA704F5%7D/LEP%20and%20Disabilities%20Subqroup%20report% 20finala.pdf 
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Center on Education Policy, 2008 

Many states have taken a "backloaded" approach to NCLB's goal of all students scoring 

"proficient" 



Focus 

Examines the annual measurable objectives for student achievement set by all 50 states 
and the District of Columbia. These objectives specify interim performance targets on the 
way toward reaching the NCLB goal of 100% of students performing at the proficient level 
on state tests by the end of school year 2013-14. 

Methodology 

Analyzed state accountability plans posted on the U. S. Department of Education Web site 
or information retrieved directly from state department of education Web sites. Data were 
retrieved in J anuary 2008. 

Major Findings 

• Backloaded approach. Twenty-three states have "backloaded" their trajectories for 
reaching 100% proficiency. In other words, they expect smaller achievement gains in 
the earlier years of NCLB and much steeper gains in later years, as 2014 grows 
closer. Some states expect leaps of 10 or more percentage points per year in the 
later years. 

• Incremental approach. Another 25 states and D.C. have adopted a more 
incremental approach that assumes steadier progress toward the 100% proficient 
goal. 

• Blended approach. The two remaining states used blended trajectories that do not 
fit readily into the backloaded or incremental categories. 

• Implications of backloaded approach. Schools and districts in backloaded states 
are likely to have more difficulty making AYR as time goes on and could see a rise in 
the number of schools identified for NCLB improvement. 

Where to Obtain 



www.cep-dc.orq 
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Center on Education Policy, 2007 

No Child Left Behind at five: A review of changes to state accountability plans 



Focus 

Examines changes to NCLB accountability plans that were approved by the U.S. Department 

of Education in 2006. 

Methodology 

Reviewed decision letters to states from the U.S. Department of Education posted on ED's 

Web site between January 1 and December 31, 2006. The decision letters reported changes 

to state accountability plans that were approved by ED (but not changes that were denied). 

Major Findings 

• Compliance with testing systems. Many changes requested were related to 
meeting the 2005-06 deadline for implementing tests that fulfilled NCLB 
requirements. Many states had to change cut scores and AYR targets because they 
had introduced new testing programs or tests at new grade levels. 

• Deadline for AYP notification. Many states did not meet the NCLB requirement to 
report the AYP status of schools and districts before the start of the school year (in 
this case, school year 2006-07). These delays created confusion about which schools 
would have to undertake the improvement steps and interventions required by NCLB. 

• Flexibility in AYP determinations. ED approved changes in 2006 that in effect 
made it easier for schools and districts to demonstrate AYP. These included, among 
others, the adoption of confidence intervals, indexing systems, and more lenient 
policies for counting scores from retests. 

• Learning from other states. The flexibility permitted by ED in 2006 does not break 
new ground. Rather, more states are copying changes that ED had already allowed in 
other states or are applying the adjustments and flexibility described in policy 
guidance issued by Secretary of Education Margaret Spellings in 2005. 

Where to Obtain 

http://www.cep-dc.orq 
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Center on Education Policy, 2006 

From the capital to the classroom: Year 4 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2004-05 and 2005-06. Summarized below 
are the study's findings about accountability and AYP issues. 

Methodology 

Collected data through a survey of all 50 states, a nationally representative survey of 299 
school districts, case studies of 38 geographically diverse districts and 42 schools, three 
national forums, and six special analyses of critical issues in implementing NCLB. 

Major Findings 

• AYP and schools identified for improvement. About 16% of all schools and 24% 
of all school districts did not make adequate yearly progress based on 2004-05 
testing. For the 2005-06 school year, about 14% of Title I schools, or 6,748 schools, 
were identified for improvement, corrective action, or restructuring. This was similar 
to the percentage identified the previous year, although the schools were not always 
the same schools. About 13% of school districts were in improvement. 

• Impact of policy changes. The stable numbers of schools in improvement were 
partly due to changes in federal and state rules that made it easier for districts and 
schools to make AYP. Examples include the use a statistical technique called 
confidence intervals to create a sort of margin of error for AYP calculations; the use 
of index systems to give credit for gains by lower-achieving students; and increases 
in the minimum number of students that must be in a subgroup for that subgroup's 
test scores to count for AYP. Other federally-approved state policy changes may have 
resulted in more students being counted as proficient, such as testing some students 
with disabilities against modified or alternate standards and counting passing scores 
from students who retake a test they previously failed. 

• Greater impact on urban districts Although all school districts have been affected 
by NCLB, urban districts have experienced the greatest effects. The majority (54%) 
of Title I schools identified for improvement for school year 2005-06 were located in 
urban districts, even though just 27% of Title I schools were located in urban 
districts. About 90% of the schools in restructuring, the most serious stage of NCLB's 
sanctions, were in urban districts. The diversity and poverty of urban districts were 
major reasons for this disproportionate impact of NCLB. Some urban districts must 
make AYP for up to 10 racial-ethnic and demographic subgroups, while some rural 
districts have to show progress for just two subgroups (white and low-income 
students). In addition, urban districts, due to their size, must make AYP for dozens 
of schools, while a small district may have just one elementary, one middle, and one 
high school. Increases in states' minimum subgroup sizes also help smaller districts 
more than larger ones. 

Where to Obtain 

http: //www.cep-dc.orq 
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Center on Education Policy, 2005 

From the capital to the classroom: Year 3 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2003-04 and 2004-05. Summarized below 
are the study's findings about accountability and AYP issues. 

Methodology 

Collected data through a survey of 49 states, a nationally representative survey of 314 
school districts, case studies of 36 geographically diverse districts and 37 schools, three 
national forums, and four special analyses of critical issues in implementing NCLB. 

Major Findings 

• Students with disabilities and ELLs. States and districts most often cited the 
accountability requirements for students with disabilities and English language 
learners as their greatest challenge. Although ED relaxed the requirements 
somewhat for both subgroups during 2004, many state and district officials indicated 
that the changes did not go far enough and viewed the testing policies for these 
groups as unfair, unrealistic, inappropriate, or instructionally meaningless. 

• Unrealistic 100% goal. Many state and local respondents saw the goal of 100% of 
students performing at proficient levels by 2014 as unrealistic and expressed doubt 
about their ability to meet state AYP targets. 

• Flexibility and challenges. Changes in federal and state policies offered more 
flexibility in NCLB implementation, which made it easier for both schools and districts 
to make AYP, at least in the short term. But states and districts still reported 
problems with NCLB accountability requirements, such as conflicts with pre-existing 
state accountability systems and difficulties meeting the deadline for reporting 
schools' AYP status before the beginning of the school year. 

• Large and urban districts. Over time, schools identified for NCLB improvement 
have become more concentrated in very large school districts and in urban areas. 
These types of districts tend to be more diverse, with more subgroups of students 
that must demonstrate AYP. 

Where to Obtain 

http: //www.cep-dc.orq 
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Center on Education Policy, 2005 

States test limits of federal AYP flexibility 



Focus 

Examines changes to NCLB accountability plans approved by the U.S. Department of 
Education in 2005. 

Methodology 

Reviewed decision letters to states from the U.S. Department of Education posted on ED's 
Web site between January and October, 2005. The decision letters reported changes to 
state accountability plans that were approved by ED (but not changes that were denied). 

Major Findings 

• AYP flexibility. In 2005, the U.S. Department of Education granted states additional 
flexibility in how they determined adequate yearly progress for NCLB. Many of the 
changes to state accountability plans approved in 2005 had the effect of making it 
easier for schools to make AYP, including the following: 

> Confidence intervals, a statistical technique intended to make allowances for 
natural fluctuations in test scores unrelated to student learning 

> Performance indexes that allow schools to get "partial credit" for improved 
performance of students below the proficient level 

> Retesting, which allows students to retake a different version of the same test 
and permits schools to use a student's best score to count toward AYP 

> Increased minimum subgroup sizes, which mean that in many schools, 
subgroups do not get counted for AYP purposes 

• Additional flexibility. In 2005, ED also permitted flexibility in other areas, such as 
allowing a few districts to provide supplemental educational services to students 
before offering them school choice and issuing new rules that allow more students 
with disabilities to be tested against modified or alternate standards. Other state 
changes in accountability plans made necessary adjustments in response to states' 
difficulties in administering the law. 

• Less transparency and comparability. Over time, changes in state accountability 
plans for NCLB have increased the complexity and variety of state AYP formulas, 
making it harder for parents and others to understand what it means if a school does 
not make AYP. In addition, constant changes in AYP formulas make it difficult to tell 
whether student achievement is really improving, because percentages proficient or 
percentages of schools missing AYP may not be truly comparable from year to year. 

• ED delays and limited information. ED often delayed in posting decision letters to 
states on its Web site and provided very little information about requested changes 
that were rejected or the rationales for accepting or rejecting changes. 

Where to Obtain 

http: //www.cep-dc.orq 
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The Civil Rights Project at Harvard University, 2006 

School accountability under NCLB: Aid or obstacle for measuring racial equity? 



Focus 

Reviews federal and state implementation of the NCLB accountability requirements and their 
impact on minority and low-income students. 

Methodology 

Examined trends in the number and types of schools identified for improvement under NCLB 
over five years (2002-03 to 2006-07). Compared the demographic characteristics of 
students in schools identified for improvement and those in schools that made adequate 
yearly progress in six states with large proportions of minority and low- income students, 
(Arizona, California, Georgia, Illinois, New York, and Virginia). Data were drawn from the 
school achievement report cards required by NCLB and other reports issued by state 
departments of education. 

Major Findings 

• Shortcomings of AYP mechanism. As it stood in 2006, the AYR mechanism used 
to identify poorly performing schools was not working effectively to encourage 
improvement in these schools and proficiency in all schools. AYP did not seem to 
serve as a preventative measure. 

• Disproportionate impact on minority and low-income students. Schools most 
likely to be identified as needing improvement are highly segregated and enroll a 
disproportionate share of a state's minority and low-income students. The NCLB 
sanctions are concentrated on schools serving disadvantaged and minority students, 
whether or not those schools are making progress that isn't measured by AYP. 

• State accountability changes. The changes that the U. S. Department of 
Education has approved to state accountability plans, which were intended to reduce 
the number of schools identified for improvement, have complicated understanding 
of what accountability means and have made comparisons from one year to the next 
meaningless. 

Where to Obtain 

http://www.civilriqhtsproiect.ucla.edu/research/esea/NCLB Policy Brief Final.pdf 
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The Civil Rights Project at Harvard University, 2006 

The unraveling of No Child Left Behind: How negotiated changes transform the law 



Focus 

Documents changes states made to their accountability plans. Examines how these policy 

shifts have affected the meaning of accountability and who benefits (and loses) from them. 

Methodology 

Reviewed decision letters from the U.S. Department of Education to all 50 states outlining 

the changes approved by ED through December 2005. 

Major Findings 

• Compromises in accountability plans. In 2004 and 2005, ED made extensive 
compromises by allowing states to make a wide variety of changes in their 
accountability plans. These changes reflected a political strategy to respond to 
growing opposition to NCLB by providing relief from some of its provisions. Many of 
the changes simply reduced the number of schools and districts identified for 
improvement without requiring any educational improvement. 

• Types of changes. Several of these policy changes affected how students with 
disabilities and English language learners were counted for accountability purposes, 
revised how states calculated test participation rates, and relaxed the highly qualified 
teacher requirements. Other state- initiated changes to accountability plans were 
negotiated on a state- by- state basis, with no guidelines about the types of changes 
states could request, no information on how the requests would be judged, and no 
guarantees that changes approved in one state would be approved in another. The 
state- initiated amendments were extensive, and many changed how states 
determined AYR. These changes have complicated the meaning of AYR and obscured 
the ability of states, districts, and schools to show improvements in student 
performance. 

• Lack of uniformity. Since the number and kinds of changes that states have 
adopted are not uniform across states, accountability no longer has a common 
meaning across states or even within states. 

• Winners and losers. There were clear winners and losers from the changes. Some 
of the changes, such as the change in the method used to identify districts for 
improvement, made it harder for some districts, primarily those serving minorities, 
to make AYR. Others, such as the changes in the highly qualified teacher 
requirements, benefited some regions of the country more than others. 

Where to Obtain 



http://eric.ed.qov/ERICDocs/data/ericdocs2sql/content_storaqe_01/0000019b/80/29/dd/3a 

.Pdf 
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The Civil Rights Project at UCLA, 2008 

Holding NCLB accountable: Achieving accountability and equity 



Focus 

Evaluates the efficacy of NCLB's test- based accountability system and discusses options for 
the law. 

Methodology 

Solicited chapters from noted education scholars about major aspects of NCLB's 
performance- based accountability system. 

Major Findings 

• Shortcomings of current system. The NCLB accountability system is not based on 
hard evidence and does not provide information necessary to know how well 
students are actually performing or what to do to improve learning and instruction. 

• Achievement results not promising. A comparison of findings from NAEP with 
state assessment results shows that federal accountability has not systematically 
improved reading and math achievement or reduced achievement gaps. The nation 
has not focused on the kinds of serious, long-term reforms that can actually produce 
gains and narrow gaps for minority students. 

• Limited state capacity. States have limited capacity to meet NCLB requirements 
and intervene in low-performing schools on the scale demanded by the law. 

• Negative effect on reform. Many NCLB provisions— including the highly qualified 
teacher definition and the testing and accountability regulations— impede school 
reform and make it more difficult for high schools serving low- income students to do 
their work. 

• Inconsistency between dropout rates and AYP. Only about 40% of the nation's 
high schools that have high dropout rates have been identified for NCLB 
improvement. 

Where to Obtain 

http://www.civilriahtsproiect.ucla.edu/news/pressreleases/pressrelease20080109- 

book.html 
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Council of Chief State School Officers, 2008 

Statewide educational accountability systems under the NCLB Act— A report on 2008 

amendments to state plans 



Focus 

Describes the types of amendments to state accountability plans that states sought to have 
approved by the U. S. Department of Education and the decisions by ED to approve or deny 
them. These amendments would change elements of the accountability systems that all 
states must have in place to comply with NCLB. 

Methodology 

Analyzed information provided voluntarily by states to the Council of Chief State School 
Officers about amendments requested to state plans during 2007-08. State information 
included copies of state documents, e-mails, and reports of telephone conversations. 
Analyzed subsequent decision letters from ED to the states posted on the ED Web site. 

Major Findings 

• High number of requests. I n 2007-08, the number of states submitting 
amendments (49 states plus D.C.) and the number of substantive amendments 
proposed (about 275) was the highest in any single year since NCLB was enacted in 
2002. Vermont was the only state that did not submit amendments. Almost all 
decision letters were posted on ED's Web site by the end of J uly 2008. 

• Types of amendments. For 2007-08, the most frequent amendment requests 
related to the following issues (number of states requesting are in parentheses): 

> Continuing or initiating flexibility to use "modified" student academic 
achievement standards in AYR decisions for certain students with disabilities (30) 

> Implementing science assessments (17) 

> I ncluding formerly served students with disabilities in AYR calculations for the 
disabilities subgroup (15) 

> Defining and calculating graduation rates for AYR decisions (16) 

> Changing the minimum subgroup size to be counted for AYR purposes (11) 

> Modifying how new schools and reorganized schools are defined for AYR (10) 

> Modifying how AYR determinations are made for schools not covered by state 
assessments and for small schools (10) 

> Modifying or clarifying the methods for calculating AYR in general (10) 

> Rermitting states to use a growth model to determine AYR (10) 

> Modifying starting points, annual measurable objectives, and interim goals on 
the way to reaching NCLB's ultimate goal of 100% proficient 

> Modifying or clarifying how the achievement of English language learners is 
included in AYR determinations or measures of English language acquisition (10) 

• Denied requests. A few states sought amendments that ED has consistently 
denied, such as limiting schools identified for improvement to those that missed AYR 
in the same subject and subgroup for two consecutive years, or targeting sanctions 
in identified schools only to those subgroups that missed AYR. 

Where to Obtain 

http://www.ccsso.orq/publications/details.cfm7Rublicationl D=369 
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Economic Policy Institute, 2008 

Grading education: Getting accountability right 



Focus 

Discusses why the narrow test- based accountability systems developed for NCLB cannot 
determine whether schools are performing satisfactorily or support interventions that ensure 
improvement. Proposes an alternative system for holding schools accountable. 

Methodology 

Commissioned surveys to determine what value the public and elected representatives place 
on the various goals of a well-designed accountability system. Interviewed teachers around 
the country about the effects of accountability policies on their instructional practices. 
Reviewed accountability plans in public and private fields. 

Major Findings 

• Goals of accountability system. According to surveys of the public and elected 
representatives, more than half of the weight of an accountability system should be 
devoted to the academic goals of basic knowledge and skills, critical thinking, 
appreciation of the arts and literature, and acquisition of occupation-specific 
technical skills, while the balance should be devoted to citizenship, social skills, and 
other physical and emotional health behaviors. Holding schools accountable for math 
and reading test scores has created incentives for educators to pay less attention to 
other curricular areas and goals for which they are not held accountable. 

• Flaws of current accountability systems. Accountability policies for NCLB and 
state systems ignore the normal variation in student abilities and include such 
fanciful proficiency definitions that even the highest-scoring countries don't come 
close to realizing them. A test- based accountability system based on a fixed 
proficiency point leads to excessive concentration on students whose performance is 
slightly below that point and ignores those who are either above or far below it. It 
also creates incentives for educators to game the system. 

• Accountability in other sectors. A review of accountability plans in sectors such as 
health care, job training, welfare, and criminal justice reveals that an accountability 
plan that relies primarily on quantitative short-term measures (similar to test scores) 
without substantial qualitative evaluation will corrupt any institution. 

• Supplementing NAEP. The model of the National Assessment of Educational 
Progress in its early years suggests some elements of a sophisticated accountability 
system, in that it focused on long-term outcomes by assessing young adults as well 
as schoolchildren and it measured behavioral as well as cognitive results of 
schooling. But because NAEP's complex sampling methodology can only generate 
results at the state level (or for very large urban districts), it cannot tell 
policymakers how individual schools or other youth institutions are contributing to 
student results. Supplementing this information requires actual inspection of schools 
and other institutions of youth development. 

• Models for school inspection. American school boards have lost sight of their 
obligation to hold schools accountable for outcomes defined by the public through 
democratic procedures. Elements of a democratic school accountability system could 
be adapted from existing arrangements, such as the current system of school 
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accreditation, which is based on school inspections but does not presently provide an 
adequate means of accountability. Some other nations more successfully use school 
visitations and inspectorate models for accountability purposes. 

• Alternative accountability system. To hold schools, districts, and other youth 
development institutions accountable, information from tests of basic skills should be 
combined with a wide array of information from other sources, including tests of 
reasoning and critical thinking, and professional visitations by experienced and 
qualified educators. The federal role in such a system should include a funding 
distribution mechanism to ensure that states with limited fiscal capacity and 
relatively large numbers of disadvantaged children have sufficient resources to 
generate the expected youth outcomes. It should also include a vastly expanded 
NAEP that could give state leaders the information they need to determine if their 
youth demonstrate balanced achievement in eight broad goal areas. 

Where to Obtain 

http://www.epi.ora/publications/entrv/books_aradinq_education 



CEP, Compendium of Major NCLB Studies 



Accountabiiity & Related Issues 



Page 18 



Editorial Projects in Education (Education Week), 2009 

Schools struggle to meet key goal on accountability 



Focus 

Reports numbers of schools nationally that failed to make adequate yearly progress 

according to NCLB criteria in school year 2007-08. 

Methodology 

Analyzed data from 47 states and the District of Columbia on the number of schools making 

AYR based on tests administered in school year 2007-08. 

Major Findings 

• Number of schools failing to make AYP. Almost 30,000 U.S. schools failed to 
make AYP in school year 2007-08, an increase of 28% over school year 2006-07 in 
states with comparable data. 

• Number of schools identified for improvement. Half of the schools that failed to 
make AYP in 2007-08 missed their achievement goals for two or more years, putting 
almost one in five of the nation's public schools in some stage of NCLB improvement. 
The number facing sanctions represents a 13% increase over school year 2006-07 
for states with comparable data. 

• Restructuring schools. Based on their 2007-08 performance, 3,559 schools— 4% 
of all schools rated— are in the planning or implementation stages of restructuring for 
the current school year. This is double the number in that category a year ago. 

• State- by- state differences. While the national data suggest a steady increase in 
the number of schools failing to make AYP, state- by- state results show that states' 
policy decisions can skew the results. In South Carolina, for example, 80% of public 
schools failed to make AYP in 2007-08, the highest proportion of any state. The 
increase can be partly attributed to the addition of new schools being rated for AYP 
and to the state's decision to set challenging standards. 

• Backloaded achievement targets. Decisions by 23 states to set low achievement 
targets in the early years of NCLB and quickly-rising targets in later years have also 
contributed to sharp increases in the number of schools not making AYP. Those 
states assumed that they would be able to ramp up student achievement by 2007- 
08, but the AYP results don't reflect that. California, for example, had a dramatic 
increase in the percentage of schools failing to make AYP, from 34% in 2006-07 to 
48% in 2007-08. 

Where to Obtain 



http://www.edweek.orq/ew/articles/2008/12/18/16avp.h28.html?tkn=MOWFHI bDiD5lcW01 
CQ6vONfAXdD6SB7Rhl 8u&print^l 
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Education Sector, 2006 

Hot air: How states inflate their educational progress under NCLB 

Focus 

Discusses how states have used their standard-setting flexibility to inflate their educational 

progress under NCLB and to minimize the number of schools that face scrutiny under the 

law. 

Methodology 

Reviewed data submitted by state departments of education to the U.S. Department of 

Education. The data came from Consolidated State Performance Reports submitted n March 

2006. 

Major Findings 

• General finding. A significant number of states have used flexibility allowed by ED 
to make it seem as if their schools are doing better than they really are. Some states 
have dramatically inflated their high school graduation rates, teacher qualifications, 
school safety, and many other aspects of school performance under NCLB. 

• AYR statistical manipulations. States used "statistical games" to make their 
performance appear better than it is. These included setting large minimum 
subgroup sizes, using confidence intervals (essentially a "plus or minus band") 
around proficiency targets and safe harbor calculations, and requiring school districts 
to fall short of AYP targets at all three grade levels in the same subject before they 
fail to make AYP. 

• Proficiency and testing differences. State differences in academic content 
standards, types of tests, and cut scores on tests for proficiency have created large 
variations in percentages proficient among states. 

• Persistently dangerous schools. I n 2006, states asserted that only 28 of the 
nation's 95,000 schools were "persistently dangerous," a designation that gives 
students the right to transfer under NCLB. Only six states reported any persistently 
dangerous schools at all. 

• MOUSSE process. Most states made it relatively easy for veteran teachers to 
demonstrate they met the NCLB "highly qualified" teacher requirements by checking 
off a series of activities under the state's HOUSSE (high objective uniform state 
standard of evaluation) process. 

Where to Obtain 

http://www.educationsector.org/usr doc/Hot Air NCLB.pdf 
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Government Accountability Office, 2006 

No Child Left Behind Act: Education assistance could help states better measure progress of 

students with limited English proficiency 



Focus 

Describes the extent to which English language learners are meeting academic progress 
goals, what states have done to ensure the validity of their academic assessments and 
English language proficiency assessments, and how the U.S. Department of Education is 
supporting states' efforts to meet NCLB testing requirements for ELLs. 

Methodology 

Collected state data on the AYP status of ELLs for 2003-04 from 48 states and district- level 
data from 18 states with the greatest numbers of or growth in ELLs. Did in-depth studies of 
ELL assessment practices in five states (CA, NE, NY, NC, and TX) and interviewed officials 
from 28 states, testing companies, and ED. Reviewed documents, convened a group of 
assessment experts, and surveyed all the states by e-mail on native language assessments. 

Major Findings 

• Progress of ELLs. I n nearly two-thirds of the 48 states with sufficient data, ELLs did 
not meet annual state proficiency targets for reading or math in school year 2003- 
04. Further, in most states, these students generally did not do as well as other 
subgroups on state elementary school math tests. 

• Academic tests. Officials in the five states studied reported following generally 
accepted procedures to ensure their academic tests for ELLs were valid and reliable. 
However, the expert group questioned whether all states were assessing ELLs in a 
valid way, noting that some lacked technical expertise. According to ED peer reviews 
of assessments in 38 states, 25 states did not provide adequate evidence of validity 
or reliability. Most states offer test accommodations to ELLs, but research is lacking 
on which accommodations are effective in mitigating language barriers. Several 
states used native language or alternate assessments, which are costly to develop 
and not appropriate for all students. 

• Language proficiency tests. Many states implemented new English language 
proficiency assessments in 2006 to meet NCLB requirements, so complete 
information on validity and reliability is not yet available. A 2005 ED-funded review 
of 17 English language proficiency tests found insufficient documentation of validity. 

• ED assistance. ED has offered technical assistance to help states assess ELLs but 
has issued little written guidance about developing English language proficiency 
tests. Officials in one-third of the 33 states contacted wanted more guidance on this 
issue. States have some flexibility in how they assess ELLs, but state officials wanted 
additional flexibility to appropriately track the academic progress of these students. * 

Where to Obtain 

http://www.qao.qov/new.items/d06815.pdf 



*GAO also delivered testimony on this issue on March 23, 2007 ( http: // www.qao.qov/new.items/d07646t. pdf ). The 
testimony noted that since the original report was published, ED has initiated a partnership with states and other 
organizations to support development of valid assessment options for ELLs. 



CEP, Compendium of Major NCLB Studies 



Accountabiiity & Related Issues 



Page 21 



Government Accountability Office, 2006 

No Child Left Behind Act: States face challenges measuring academic growth that 
Education's initiatives may help address* 



Focus 

Discusses the number of states using growth models— a term that encompasses a variety of 
methods for tracking changes in test scores or proficiency levels over time. Also describes 
why growth models are used, how they can measure progress toward achieving key NCLB 
goals, and what challenges states face in using growth models to meet the law's key goals. 

Methodology 

In 2006, GAO surveyed all states and conducted site visits in California, Massachusetts, 
North Carolina, and Tennessee. In Massachusetts and Tennessee, GAO analyzed student- 
level data from selected schools. 

Major Findings 

• States with growth models. As of March 2006, 26 states were using growth 
models as part of their state education accountability systems, and 22 were either 
considering or in the process of implementing them. Most states using growth 
models measured progress for schools and for student subgroups, and seven states 
also measured growth for individual students. States used growth models to target 
resources for students that needed extra help or to award teachers bonuses based 
on school performance. 

• Purposes of growth models. Certain growth models are capable of tracking 
progress toward 100% proficiency by 2014 and progress in closing achievement 
gaps. Massachusetts used its model to set targets based on the growth that it 
expected from schools and subgroups. Tennessee's model projects students' test 
scores and whether students will be proficient in the future. 

• Challenges. States faced challenges measuring academic growth, such as creating 
data and assessment systems to support growth models and having sufficient 
personnel to analyze and communicate results. The use of growth models to 
determine AYP may also challenge states to make sure that students in low- 
performing schools receive needed assistance. 

• Pilot program. U. S. Department of Education initiatives may help states address 
these challenges. In November 2006, ED started a pilot project to allow states to use 
growth models to determine adequate yearly progress; these models had to meet 
specific criteria, such as being able to track progress of individual students. ED also 
provided grants to states to track individual test scores over time. 

Where to Obtain 

http: // www.qao.qov/ new. items/d06661. pdf 



‘GAO also delivered testimony on this issue on J uly 27, 2006 (http://www.gao.gov/new.items/d06948t.pdf). 
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Government Accountability Office, 2005 

No Child Left Behind Act: Education could do more to help states better define graduation 
rates and improve knowledge about intervention strategies 



Focus 

Examines the definitions of graduation rates that states use for NCLB purposes, the actions 
the U.S. Department of Education has taken to help states meet legal requirements, the 
factors that affect the accuracy of states' graduation rates, and ED's role in ensuring 
accurate data. Also discusses interventions that could increase graduation rates and 
assesses ED's efforts to disseminate intervention research. 

Methodology 

Surveyed 49 states and Puerto Rico. Also conducted a case study in one state to calculate 
graduation rates, site visits in three states to review data accuracy, site visits in six states 
to observe interventions and interview staff, and telephone interviews in 20 states to get 
information about definitions used, implementation status, and ED guidance. 

Major Findings 

• Cohort definition. In 2005, 12 states determined graduation rates for purposes of 
NCLB by using the cohort definition, which tracks students from the time they enter 
high school until the time they leave. Thirty-two states used a definition based 
primarily on the number of graduates and the number of dropouts over a four-year 
period. The remaining states used other definitions. Because the cohort definition is 
more precise, most states that were not using it planned to do so once they 
developed the capability to track students over time, a capability that many state 
data systems did not have at the time of this study. 

• ED guidance. ED assisted states with definitions of graduation rates primarily on a 
case- by-case basis, but had not provided guidance to all states on ways to account 
for selected students, such as students with disabilities. As a result, state approaches 
for calculating graduation rates were not consistent. 

• Accuracy. Student mobility was the primary factor affecting the accuracy of 
graduation rates. Another factor was whether states verified student data; fewer 
than half the states conducted audits of data used to calculate graduation rates. ED 
had taken steps to help states address data accuracy issues, but ED officials said 
they could assess only those state systems that had been in place for a while. 

• Promising interventions. Many interventions were being used to raise graduation 
rates, but few had been rigorously evaluated. Based on its state and school visits, 
GAO identified five interventions that had been rigorously evaluated and showed 
potential for improving graduation rates, such as Project GRAD. Other interventions 
in schools visited by GAO were considered promising by experts and officials. 

• Lack of ED dissemination. As of 2005, the U.S. Department of Education had not 
acted on GAO's 2002 recommendation that it evaluate intervention research, which 
ED agreed with, and had done little to disseminate this type of research. 

Where to Obtain 

http://www.qao.qov/new.items/d05879.pdf 
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Great Lakes Center for Education Research and Practice, 2005 

The impact of the adequate yearly progress requirement of the federal "No Child Left 
Behind" Act on schools in the Great Lakes region 



Focus 

Examines the implementation of the NCLB accountability and adequate yearly progress 

requirements in the six states of the Great Lakes region (Illinois, Indiana, Michigan, 

Minnesota, Ohio, and Wisconsin). 

Methodology 

Analyzed the AYP status of schools in the Great Lakes states and the annual objectives 

schools must meet to make AYP. 

Major Findings 

• Assumptions for steep increases in later years. The annual objectives for four of 
the six Great Lakes states assumed incremental increases in the percentage of 
students performing at the proficient level for the early years of NCLB 
implementation. However, these are followed by steep, backloaded objectives that 
assume very large increases in performance in the later years. 

• Likelihood of widespread failure to make AYP. None of the Great Lakes states 
was among the top ten states with the highest percentage of schools failing to make 
AYP in either 2003 or 2004. By 2014, however, approximately 85% of schools in the 
Great Lakes states were projected to fail to make AYP under the most optimistic 
scenarios. Under more realistic circumstances, the overall failure rate is projected to 
be at or above 95%. 

• Concerns about NCLB accountability. The authors of the study questioned the 
sustainability of the AYP requirements and cautioned that schools are not capable of 
closing the achievement gap without resolving underlying social problems. Adequate 
funding for remediation and social infrastructure is essential to meeting the goals of 
NCLB. 

Where to Obtain 

http://www.qreatlakescenter.orq/q I new doc/EP5L-0505-109-EPRU. Great lakes.pdf 
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RAND, 2007 

Standards- based accountability under No Child Left Behind: Experiences of teachers and 

administrators in three states 



Focus 

Describes how NCLB- related accountability policies have influenced attitudes and actions at 
the district, school, and classroom levels in three states, with a focus on math and science. 

Methodology 

Analyzed data from large-scale surveys and interviews and small-scale case studies 
conducted during school years 2003-04, 2004-05, and 2005-06 in California, Georgia, and 
Pennsylvania. Information was gathered from teachers and superintendents in a 
representative sample of 27 school districts in the three states and in 125 randomly 
selected elementary and middle schools within those districts. Site visits were conducted in 
14 schools in 2003-04 and 16 schools in 2004-05. The report focuses on 2004-05 data. 

Major Findings 

• State differences. State accountability systems developed in response to NCLB 
differed across the three states in terms of the content of their academic standards, 
difficulty of performance standards, methods for calculating adequate yearly 
progress, trajectories for reaching 100% proficiency, support and technical 
assistance mechanisms, and other aspects. Many of these differences were related to 
state policies in place before NCLB, such as the degree of state involvement in 
standards- based accountability. 

• School improvement activities. Despite state differences, districts and schools in 
the three states responded to accountability actively and in broadly similar ways. The 
school improvement activities described as most important by superintendents 
included aligning curriculum with state standards and assessments, using data for 
decision making, and providing extra support to low-performing students. In addition 
to these actions, most superintendents reported providing technical assistance to 
help schools improve and offering professional development for principals and 
teachers; a large number of principals reported providing extra learning 
opportunities for low-performing students. Other common activities included 
implementing test preparation and giving interim tests. District actions appeared to 
be influenced by the specific content and features of state standards and 
assessments. 

• Teacher views. Teachers noted that NCLB had influenced their instruction in some 
beneficial ways, including changes to align instruction with standards and improve 
their own practices. Teachers also described some less desirable responses to NCLB, 
such as narrowing curriculum and instruction toward tested topics or certain problem 
formats and focusing more on "bubble kids" who were already closest to reaching 
proficiency. Teachers also expressed concern about negative effects of accountability 
requirements on learning for high-achieving students. 

• Specific concerns. Most superintendents, principals, and teachers expressed 
support for the idea of standards- based accountability, but these groups had 
different concerns about specific features of accountability systems. Most 
administrators, but only a small minority of teachers, thought state test scores 
accurately reflected student achievement. Administrators were more likely than 
teachers to think the pressure of accountability led to improved curriculum and 
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learning. Teachers expressed concerns about reduced morale and other negative 
effects on their teaching. Still, teachers reported an increased focus on achievement 
in their schools as a result of NCLB, as well as greater coordination of and rigor in 
the curriculum. 

• Adverse effects. Both teachers and administrators identified factors they believed 
adversely affected their efforts to meet NCLB goals, including inadequate funding, 
insufficient numbers of highly qualified teachers, and insufficient instructional and 
planning time. Teachers said that students' lack of basic skills, inadequate parent 
support, and student absenteeism and tardiness hampered their efforts to promote 
high achievement. 

Where to Obtain 

http://www.rand.orq/pubs/monoqraphs/2007/RAND MG589.pdf 
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Thomas B. Fordham I nstitute, 2009 

The accountability illusion 



Focus 

Examines the NCLB accountability systems and the basic adequate yearly progress rules as 
they operate in practice in 28 states. 

Methodology 

Evaluated student performance in 18 elementary schools in 28 states and 18 middle schools 
in 26 states; the schools varied by size, achievement, diversity, and other factors. Analyzed 
whether a school in a particular state would make AYP according to the proficiency cut 
scores and 2008 annual AYP targets of other states, and if not, what factors within NCLB 
explained the difference. As part of the AYP analysis, applied confidence intervals according 
to each state's rules and evaluated the performance of all subgroups within a school that 
met or exceeded each state's minimum subgroup size requirements. 

Major Findings 

• Different results using different states' AYP rules. Within the elementary school 
sample, the number of schools that made AYP varied greatly by state. Almost all of 
the schools in the sample failed to make AYP in some states, and nearly all of these 
same schools made AYP in others. In Massachusetts, for example, a state with high 
proficiency cut scores and relatively challenging annual targets and AYP rules, only 1 
of 18 elementary schools made AYP; in Wisconsin 17 schools made AYP. 

• More consistency but lower performance in middle schools. There was more 
consistency in AYP results across states among the middle schools in the sample 
because so few of these schools made AYP in any state. I n 21 of the 26 states 
studied, two or fewer middle schools made AYP. I n no state did even half of the 18 
middle schools meet the 2008 AYP requirements. This was mostly because middle 
schools tended to enroll more students than elementary schools and generally had 
subgroups that were large enough to count separately for accountability purposes. 
Although the racial-ethnic subgroups in the sample schools met annual targets in 
many states, the subgroups of students with disabilities and English language 
learners, when large enough to count, failed to meet AYP targets in almost all of the 
schools in nearly every state. 

• Importance of minimum subgroup size rules. State rules for minimum subgroup 
size (called n size) were critical in determining whether subgroup performance would 
hurt a school's chances of making AYP. For example, the highest performing middle 
school in the sample had strong performance overall and showed growth in students' 
performance over time, but it failed to make AYP in 21 of the 26 states because of 
subgroup performance. The states where this school did make AYP had larger n 
sizes. Generally, the lower the state's n size, the more subgroups for which the 
typical school is accountable, and the more separate targets a school must hit. 

Where to Obtain 

http://www.edexcellence.net/index.cfm/news the- accountability- illusion 
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Thomas B. Fordham I nstitute, 2007 

The proficiency illusion 



Focus 

Examines the relative rigor of states' proficiency standards (cut scores students must reach 
on state tests to be considered proficient) and whether some states' tests are harder to 
pass than others. Considers whether state proficiency standards have changed since NCLB 
was enacted and whether standards in different grades are equivalent in difficulty. 

Methodology 

Estimated the differences among states in proficiency cut scores by analyzing test data from 
schools in 26 states whose pupils participated in both state tests and assessments 
developed by the Northwest Evaluation Association (NWEA). 

Major Findings 

• Wide variation among states. State tests varied greatly in difficulty. By the 
study's estimates, state cut scores for proficient performance ranged from the 
percentile on the NWEA scale to the 77*^^ percentile. 

• Changes in test difficulty. Most state tests had not changed in difficulty in recent 
years. Still, cut score estimates in reading and/or math declined significantly in at 
least two grades in eight states and rose in just four states. 

• Differences between reading and math. Math tests were consistently more 
difficult to pass than reading tests. In seven states, the difference in cut scores 
between 8*^^ grade math and reading was more than 10 percentile points, which could 
create a misimpression that students are doing better in reading when they aren't. 

• Cut scores and pass rates. I mprovements in student proficiency rates on state 
tests can largely be explained by declines in the estimated cut scores for those tests. 

• Differences in cut score by grade level. Eighth-grade tests were consistently and 
dramatically more difficult to pass than those in earlier grades (even after taking into 
account obvious differences in subject-matter complexity and children's academic 
development). In three states, differences between 3'”'^ and grade cut scores in 
reading were 20 or more percentile points. Across states, this situation could give a 
possibly false impression that elementary schools are performing at much higher 
levels than middle schools. 

• Lack of meaning of proficiency. Five years into implementation of NCLB, there 
was no common understanding of what "proficiency" meant. Its definition varies 
from state to state, from year to year, from subject to subject, and from grade level 
to grade level. This suggests that the goal of achieving "100 percent proficiency" has 
no coherent meaning, either. 

Where to Obtain 

http://www.edexcellence.net/doc/The Proficiency lllusion.pdf 
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Thomas B. Fordham I nstitute, 2006 

2006: The state of state standards 



Focus 

Rates the quality of state academic content standards in all states and examines the 

relationship of the standards ratings to state achievement on NAEP. 

Methodology 

Selected experts to review and give quality grades to the standards of the 50 states and the 

District of Columbia. Standards were examined in five subjects: U.S. history (2003), 

English/language arts (2005), mathematics (2005), science (2005), and world history 

(2005). Compared each state's rating for its standards with its performance on NAEP. 

Major Findings 

• Ratings of standards. State standards as a whole were no better in 2006 than in 
2000. The average grade given by the study's standards raters was a C- minus, the 
same as in 2000. Two-thirds of U.S. students attended school in states with 
academic standards in the C, D, and F range. Still, several states have shown 
marked progress in the quality of their standards, especially Indiana, New York, 
Georgia, and New Mexico. But other states have made their standards worse, 
including Utah, Nebraska, New Hampshire, and Wisconsin. 

• Link between strong standards and NAEP gains. According to several indicators, 
states with strong standards have made gains on the NAEP assessments. 

• Reading. In grade 4 reading, 10 states made statistically significant gains from 
1998 to 2005 in the percentage of students performing at the proficient level on 
NAEP. Nine of those 10 states received a grade of C or above from the raters for 
their English/language arts standards. 

• Science. Five states made significant gains on the science NAEP between 2000 and 
2005 at both the and grade levels. Three of the five states had the best 
science standards in the nation, according to the raters. 

• Math. The relationship between the quality of standards and NAEP scores was less 
clear in mathematics, although four of the six states that received "honors" grades 
from the raters also posted statistically significant gains on the grade 8 NAEP from 
2000 to 2005, either for the state as a whole or for their poor and minority students. 
However, many other states made progress, too. 

Where to Obtain 

http://www.edexcellence.net/detail/news.cfm7news id=358&id=130 
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U.S. Department of Education, National Center for Education Statistics, 2007 

Mapping 2005 state proficiency standards onto the NAEP scales 



Focus 

Compares the relative stringency of states' academic standards by mapping state standards 
for proficient performance on state reading and mathematics tests onto the appropriate 
scoring scale of the National Assessment of Educational Progress (NAEP). 

Methodology 

Developed a complex methodology for mapping states' cut scores for proficient performance 
(called "proficiency standards" in the report) at grades 4 and 8 for school year 2004-05 onto 
a common scale— the NAEP scoring scale. Compared the results of this mapping with states' 
average scores on the 2005 NAEP. The analysis included 32 states for grade 4 reading, 34 
states for grade 8 reading, 33 states for grade 4 math, and 36 states for grade 8 math. 

Major Findings 

• Differences in stringency of state standards. There is a strong negative 
correlation between the proportions of students meeting states' proficiency standards 
and the NAEP score equivalents to those standards. In other words, states with 
higher percentages of students scoring proficient on state tests tend to have lower 
NAEP score equivalents when their proficiency standards are mapped onto the NAEP 
scale. This suggests that differences among states in percentages proficient can be 
largely attributed to differences in the stringency of their standards. 

• Comparison of state proficiency standards and NAEP scores. There is, at best, 
a weak relationship between the NAEP score equivalents for a state's proficiency 
standard and its average scores on NAEP. 

• Difference in state and NAEP definitions of proficient. NAEP has its own cut 

scores, or standards, that define various achievement levels on its assessments. 
When state proficiency standards were mapped onto the NAEP scoring scale, the 
resulting score equivalents for most states fell below the NAEP cut score for 
"proficient" performance, and in many states fell below the NAEP cut score for 
"basic" performance. In other words, state definitions of "proficient" performance are 
often less challenging than the NAEP standard for proficient performance and are 
sometimes less challenging than the NAEP "basic" standard. 

Where to Obtain 

http://nces.ed.aov/nationsreportcard/pdf/studies/2007482.pdf 
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U.S. Department of Education, Policy and Program Studies Service, 2007 

National Assessment of Title I final report: Vol. I, implementation* 



Focus 

Comprehensive study that describes the progress of states, districts, and schools through 
school year 2004-05 in implementing key provisions of Title I. Summarized below are the 
study's findings about accountability and adequate yearly progress. 

Methodology 

Drew on data from a set of implementation studies by the U.S. Department of Education. 
Data for these studies came from surveys conducted in a nationally representative sample 
of school districts, other state and local surveys, state performance reports, and the 
National Assessment of Educational Progress. 

Major Findings 

• Assessments. During 2005-06, all states administered reading and mathematics 
assessments intended to meet NCLB requirements. As of September 1, 2007, 24 
state assessment systems had been approved as meeting all NCLB requirements 
through the U. S. Department of Education's peer review process. Most states had 
met the requirement to annually assess at least 95% of their students, although 15 
states had not met the test participation requirement for one or more subgroups. 

• Identification for improvement. For school year 2005-06, 12% of the nation's 
schools (11,648 schools) were identified for improvement based 2004-05 testing. 
Title I schools accounted for 84% of identified schools. About 32% of high- poverty 
schools and 31% of high-minority-enrollment schools were identified for 2004-05, 
compared with 4% of schools with low concentrations of these students. Urban 
schools were also more likely to be identified than other schools. 

• Adequate yearly progress. Three- fourths of all schools and districts met all 
applicable AYP targets in 2004-05 testing. Schools most commonly missed AYP 
targets for the achievement of all students and/or multiple subgroups. Of the schools 
that did not make AYP, only 21% of schools missed due to a single subgroup (most 
often students with disabilities). Schools in states with more challenging proficiency 
standards (relative to NAEP) were less likely to make AYP and had much further to 
go to reach the goal of 100% proficient by 2014 than schools in states with lower 
standards. Schools that were held accountable for more subgroups were less likely to 
make AYP. For example, 45% of schools that had to calculate AYP for six or more 
subgroups did not make AYP, compared with 5% of schools with just one subgroup. 

• Pace toward 100% proficient. Slightly more than half the states have set 
"delayed acceleration" trajectories for the pace of improvement schools are expected 
to make to reach the 100% proficient goal. Rather than expecting schools to improve 
in roughly equal increments each year, these states expect much more rapid growth 
from 2009 to 2014 than from 2004 through 2009. 

Where to Obtain 

http://ies.ed.qov/ncee/pubs/20084012/ 



‘For additional information about topics covered in this study, see the description for U. S. Department of 
Education, State and local implementation of the No Child Left Behind Act. Vol. Ill, interim report. 



CEP, Compendium of Major NCLB Studies 



Accountabiiity & Related Issues 



Page 31 



U.S. Department of Education, 2007 

State and local implementation of the No Child Left Behind Act. Vol. 1 1 1— Accountability 

under NCLB: Interim report* 



Focus 

Describes state and local implementation of the standards, assessment, and accountability 
provisions of Titles I and III of ESEA (as amended by NCLB) through school year 2004-05. 

Methodology 

Analyzed state data and documents; interviewed state officials; and conducted a survey of a 
nationally representative sample of 300 school districts and of 1,483 elementary, middle, 
and high schools in those districts. Individuals surveyed included more than 10,000 
teachers, 1,783 principals and district administrators, and 950 Title I paraprofessionals. 

Major Findings 

• State progress. States, districts and schools had mostly met the relevant NCLB 
accountability requirements through 2004-05. All states, D.C., and Puerto Rico had 
adopted achievement standards in reading and math and other required indicators. 

• States behind schedule. Twenty states were behind schedule in implementing 
assessments of English language proficiency. And 20 states were unable to notify 
schools, even preliminarily, about their test performance before September 2004. 

• State variations. State accountability systems vary significantly in such areas as 
achievement levels required for proficiency, types of assessments, and pace of 
improvement expected to reach 100% proficient in 2014. 

• Disabilities and ELL subgroups. About one-third of the schools that did not make 
AYP in 2003-04 missed targets for students with disabilities or English language 
learners. About two-thirds of the schools that missed targets for these subgroups 
reported needing technical assistance to improve instruction for these students. 

• Types of schools identified for improvement. Schools identified for improvement 
were most likely to be high-poverty, high-minority-enrollment, large urban schools— 
the schools to which Title I has historically directed substantial resources. 

• Improvement efforts. Nearly all schools reported making multiple improvement 
efforts. Schools identified for improvement focused on more areas of improvement 
than non- identified schools. Schools also reported receiving technical assistance that 
met their needs, except that about half of the schools needing assistance in serving 
students with disabilities or ELLs did not have these needs met. States and districts 
were implementing the required interventions in schools identified for improvement 
and corrective action but were not implementing the required actions in most of the 
1,199 schools in restructuring. 

Where to Obtain 

http://www.ed.aov/ rschstat/eval/disadv/ncib-accountabilitv/ nclb-accountabilitv.pdf 



‘For additional and updated information about topics covered in this study, see the description for U. S. 
Department of Education, National Assessment of Title I: Final report, Vol. I. 
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The Aspen Institute, Commission on No Child Left Behind, 2007 

The state of the achievement gap 



Focus 

Examines overall achievement for different subgroups of students and whether achievement 
gaps between subgroups have decreased, remained the same, or increased. Presents 
specific data and findings for each of the seven states included in the study. 

Methodology 

Analyzed data from state reading and mathematics tests in seven states (CA, IL, MS, NY, 

NC, OR, and TX) from 2002 through 2005. Grades 3, 5, 8, and high school were examined 
(except in NY, where grades 4, 7, and 10 were examined). Also looked at achievement gaps 
on the National Assessment of Educational Progress. 

Major Findings 

• Evidence of gaps. Achievements gaps existed in both state test and NAEP results in 
all seven states studied. These gaps could be quite large, ranging from 11 
percentage points to more than 40. Gaps were found between white and black 
students, white and Hispanic students, and students with disabilities and non- 
disabled students, and for other ethnicities including Asians and American Indians. 

• State differences. While achievement on state tests and standards was relatively 
high in the early grades in some states, such as Texas, North Carolina, and 
Mississippi, achievement decreased as students matriculated through the system. 
North Carolina and Mississippi both showed measurable gains in closing gaps in the 
earlier years, while gap closure in Texas stagnated and sometimes reversed in high 
school. Other states showed little progress in either raising achievement or closing 
gaps. 

• Mixed trends by grades and subjects. In all of the states, progress in overall 
achievement and gap closure was mixed, varying from grade to grade and subject to 
subject. 

• Gaps on NAEP. Even though overall achievement on NAEP was below achievement 
on state assessments, the achievement gaps on NAEP data were generally 
comparable to state assessments with a few exceptions. 

Where to Obtain 



http://www.aspeninstitute.org/atf/cf/%7BDEB6F227-659B-4EC8-8F84- 

8DF23CA704F5%7D/AchievementGapl.l9.07.pdf 
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Center on Education Policy, 2008 

Has student achievement increased since 2002? State test score trends through 2006-07* 

Focus 

Describes trends in scores on the state tests used for NCLB accountability. Examines trends 
in overall achievement and achievement gaps from 2002 (the year NCLB was enacted) 
through 2007. Compares state test score trends with trends in state- by- state results of the 
National Assessment of Educational Progress. Includes profiles for each state with test score 
trends and information about the state's testing system. 

Methodology 

Collected, verified, and analyzed extensive test data in reading and mathematics, through 
2006-07 where available, from all 50 states. Looked at percentages of students scoring 
proficient, as well as effect sizes, an indicator based on average test scores. Analyzed 
trends in state NAEP results for all states. Limited analysis to test results that were 
comparable from year to year and took other steps to address common problems with 
comparisons of achievement data. 

Major Findings 

• State gains. Since 2002, the percentages of students scoring proficient on state 
reading and math tests have gone up in most states with at least three years of 
comparable test data. Gains tended to be larger in elementary and middle school 
grades than in high school. Achievement has also risen in most states according to 
effect sizes. 

• NAEP and state tests. Trends in reading and math achievement on NAEP have 
generally moved in the same positive direction as trends on state tests, although 
gains on NAEP tended to be smaller. The exception was in grade 8 reading, where 
fewer states showed gains on NAEP than on state tests, especially using effect sizes. 

• State gap trends. In states with sufficient data to determine gap trends, gaps on 
state tests have narrowed more often than they have widened since 2002, 
particularly for African American and low- income students. Trends were also largely 
positive for Latino students, but this finding is less conclusive because in many states 
the Latino subgroup has changed significantly in size. On the whole, percentages 
proficient and effect sizes revealed similar trends of gaps narrowing or widening, 
although percentages proficient gave a more positive picture than effect sizes. 

• NAEP gap trends. Gaps on NAEP have also narrowed more often than they have 
widened in states with sufficient data to determine trends. The exception was in 
grade 8 math, where gaps widened more often than they narrowed for most 
subgroups. NAEP results showed less narrowing of gaps than state tests did. 

• Caution about causation. It is impossible to determine the extent to which recent 
achievement trends have occurred because of NCLB. Since 2002, many different but 
interconnected actions have been taken at the federal, state, and local levels to raise 
achievement. Furthermore, NCLB has affected all public school students, so there is 
no suitable comparison group to show what would have happened without NCLB. 



For additional information, see the summary for CEP’s 2007 achievement study. Answering the Question That 
Matters Most. 
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Where to Obtain 

http: //www.cep-dc.orq 



CEP, Compendium of Major NCLB Studies 



Achievement & Related Issues 



Page 4 



Center on Education Policy, 2007 

Answering the question that matters most: Has student achievement increased since No 

Child Left Behind?*^ 



Focus 

Examines test data from all 50 states to determine whether student achievement on state 
tests has increased and whether achievement gaps between subgroups of students have 
closed since NCLB was implemented in 2002. Includes profiles for each state with test score 
trends and information about the state's testing system. 

Methodology 

Collected and analyzed extensive test data in reading and mathematics, through 2005-06 
where available, from all 50 states for state tests used for NCLB accountability. Had states 
verify the accuracy of their data. Looked at percentages of students scoring proficient, as 
well as effect sizes, an indicator based on average test scores. Limited analysis to test 
results that were comparable from year to year and took other steps to address common 
problems with achievement data. Included pre-NCLB (1999-2002) and post-NCLB test 
results where comparable data were available. 

Major Findings 

• State test score gains. I n most states with three or more years of comparable test 
data, student achievement in reading and math has gone up since 2002, the year 
NCLB was enacted. 

• Gap trends. There is more evidence of achievement gaps between groups of 
students narrowing since 2002 than of gaps widening. Still, the magnitude of the 
gaps is often substantial. 

• Pre- and post-NCLB trends. In 9 of the 13 states with sufficient data to determine 
pre- and post-NCLB trends, average yearly gains in test scores were greater after 
NCLB took effect than before. 

• Caution about causation. It is very difficult, if not impossible, to determine the 
extent to which these trends in test results have occurred because of NCLB. Since 
2002, states, school districts, and schools have simultaneously implemented many 
different but interconnected policies to raise achievement. 

• Test data availability. Although NCLB emphasizes public reporting of state test 
data, the data necessary to reach definitive conclusions about achievement were 
sometimes hard to find or unavailable, or had holes or discrepancies. More attention 
should be given to issues of the quality and transparency of state test data. 

Where to Obtain 

http: //www.cep-dc.orq 



*For additional and updated information, see the summary for CEP’s 2008 achievement study. Has Studetn 
Achievement Increased Since 2002? 

^For additional information from state officials about the achievement trends discussed in this report and the 
impact of NCLB, see the 2007 report from the Human Resources Research Organization and Center on Education 
Policy, Behind the numbers: Interviews in 22 states about achievement data and No Child Left Behind policies. 
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Center on Education Policy and Human Resources Research Organization, 2007 

Behind the numbers: Interviews in 22 states about achievement data and NCLB 



Focus 

Reports evidence from state experts about the test score trends identified in CEP's 2007 
report, Answering the Question That Matters Most: Has Student Achievement Increased 
Since No Child Left Behind? Reports the views of state officials about the impact of NCLB 
provisions on achievement and other outcomes and their suggestions for changing the law. 

Methodology 

Interviewed state education officials in the 22 states with the longest periods of comparable 
test data before and after NCLB. 

Major Findings 

• Confirmation of trends. State officials interviewed generally confirmed the 
accuracy of the achievement data and trends reported in CEP's 2007 achievement 
study. Several provided additional information, explanations, or clarifications. 

• Limited research- based evidence. Most states had not done their own evidence- 
based studies of achievement trends or factors that might explain these trends. 

State officials said this was largely due to a lack of staff and rising demands on staff. 

• Changes in testing systems. Many states had changed their testing systems since 
2002, often in ways that made it invalid to compare test data over time. The most 
common changes were done to comply with NCLB, such as testing more grades, 
disaggregating data for subgroups, or assessing English language proficiency. Many 
states made other types of changes, such as revising the content standards on which 
tests are based, adding new tests or modifying existing tests, changing the cut 
scores defining proficient performance, and changing test contractors. 

• Future changes. Many states were pursuing additional changes in their testing 
systems that would affect the future comparability of test data, such as revising 
content standards, changing high school exams, or adopting new or additional 
assessments. Some states planned to make more test data available to the public. 

• Need for technical assistance. Many states emphasized the need for federal 
technical assistance (and often funding) to improve the capacity of states and school 
districts to collect and analyze data required by NCLB. Some states desired better 
information, guidance, or research from the federal government about best practices 
for improving achievement or closing achievement gaps. 

• Anecdotal evidence. State officials usually cited anecdotal evidence rather than 
research studies to support conclusions about the impact of NCLB on achievement. 
Several states felt that disaggregating data for subgroups had improved 
achievement. But most interviewees did not see NCLB accountability requirements as 
a very effective way to identify low-performing schools or raise student achievement. 

Where to Obtain 



www.cep-dc.orq 
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Center on Education Policy, 2006 

From the capital to the classroom: Year 4 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2004-05 and 2005-06. Summarized below 
are the study's findings about achievement. 

Methodology 

Collected data through a survey of all 50 states, a nationally representative survey of 299 
school districts, case studies of 38 geographically diverse districts and 42 schools, three 
national forums, and six special analyses of critical issues in implementing NCLB. 

Major Findings 

• Gains reported on surveys. According to the study's national surveys, 78% of 
districts reported that student achievement had improved from 2003-04 to 2004-05 
on state tests used for NCLB. During this period, 35 states said achievement had 
improved in reading, and 36 states said it had improved in math. But NAEP showed 
no gains in reading and small gains in math from 2002 to 2005. Case studies of 
school districts revealed a more mixed and complex view of achievement, with 
fluctuations in achievement from year to year or in different grades. 

• State and district views of factors contributing to gains. States and districts 
cited school district policies and programs as important contributors to achievement 
gains more often than they cited NCLB requirements. About three-fourths of the 
states responding to CEP's survey rated district policies as "important" or "very 
important" causes of increased student achievement, and most also rated state 
policies as important or very important. About 79% of the districts surveyed rated 
their own policies as important or very important causes, far more than those 
crediting federal policies for gains. However, about half of the district officials 
surveyed cited NCLB's adequate yearly progress requirements as an important factor 
in rising achievement— a view echoed by two-thirds of the states with rising 
achievement. 

• Effect of regulatory changes. While evidence from the study suggested that 
increased learning accounted for some of the gains in state test results, many states 
have also taken advantage of additional flexibility from ED to make policy changes 
that may have resulted in more students being counted as proficient. It is not clear 
to what extent state policy changes have contributed to rising percentages of 
students reaching proficiency. 

• Achievement gaps. More than two-thirds of the states reported that achievement 
gaps between student subgroups were narrowing or staying the same in math, and 
about four-fifths of states said gaps were narrowing or staying the same in reading. 
Similarly, more school districts said that gaps were narrowing or staying the same 
than said that gaps were widening. 

Where to Obtain 



http: //www.cep-dc.orq 
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Center on Education Policy, 2005 

From the capital to the classroom: Year 3 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2003-04 and 2004-05. Summarized below 
are the study's findings about achievement. 

Methodology 

Collected data through a survey of 49 states, a nationally representative survey of 314 
school districts, case studies of 36 geographically diverse districts and 37 schools, three 
national forums, and four special analyses of critical issues in implementing NCLB. 

Major Findings 

• Gains reported on survey. Student achievement was improving on the state tests 
used for NCLB according to 73% of the states and 72% of the school districts 
surveyed by the study. States and districts were also more likely to report that 
achievement gaps between white and African American students, white and Hispanic 
students, and English language learners and non-ELL students were narrowing rather 
than widening or staying the same. Testing experts caution, however, that these 
achievement gains should be considered preliminary rather than definitive because 
high-stakes testing and accountability programs can cause early spikes in state test 
scores that do not persist over time or do not show up to the same extent on other 
tests. 

Where to Obtain 



http: //www.cep-dc.orq 
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The Civil Rights Project at Harvard University, 2006 

Tracking achievement gaps and assessing the impact of NCLB on gaps 



Focus 

Anaiyzes trends in scores on the Nationai Assessment of Educationai Progress tests in 
reading and mathematics for pubiic schooi 4*^^ and graders of different raciai and 
socioeconomic groups. Looks the pre-NCLB (1990-2001) and post-NCLB (2002-2005) 
periods. 

Methodology 

Used NAEP's nationai- ievei and state- ievei aggregate measures of performance in scaie 
scores. Aiso used the percentages of students scoring at or above the NAEP proficient ievei. 
Drew from the 1990-2005 NAEP pubiic schooi sampie for the NAEP reading and math 
assessments at grades 4 and 8. 

Major Findings 

• Overall impact of NCLB. NCLB did not have a significant impact on improving 
reading and math achievement across the nation and states. The nationai average 
scores on NAEP remained fiat in reading and grew at the same pace in math after 
NCLB as they did before, in grade 4 math, there was a temporary improvement right 
after NCLB, but it was foiiowed by a return to the pre-reform growth rate. 

• Proficiency on NAEP. if the trend through 2005 were to continue, oniy 24% to 
34% of students wouid meet the NAEP definition of proficiency in reading by 2014, 
and 29% to 64% wouid meet it in math. 

• Gap trends. NCLB has not heiped the nation and states significantiy narrow the 
achievement gap. The raciai and socioeconomic gaps in NAEP reading and math 
achievement have persisted after NCLB. There was some reduction in gaps in math 
right after NCLB, but the progress was not sustained, if the trend through 2005 were 
to continue, the proficiency gap between advantaged white and disadvantaged 
minority students wouid not ciose by 2014, and the percentage of poor and biack 
students meeting the NAEP proficiency definition wouid be 25% in reading and iess 
than 50% in math. 

• Lack of scale-up. NCLB's attempt to scaie up the aiieged success of states that 
adopted test-driven accountabiiity poiicies before NCLB did not work, it did not 
enhance academic improvement in the first generation of states with test-driven 
accountabiiity (such as Fiorida, North Caroiina, and Texas), nor did it transfer the 
effects to a second generation of states. Both first and second generation states 
faiied to narrow NAEP reading and math gaps after NCLB. 

• I nflated gains. The state assessment resuits that are the basis of NCLB 
accountabiiity are misieading because they tend to significantiy infiate proficiency 
ieveis and proficiency gains and to defiate achievement gaps. The higher the stakes 
of the state assessments, the greater the discrepancies between state and NAEP 
resuits. 

Where to Obtain 

http://www.civiiriahtsproiect.ucia.edu/research/esea/ncib_naep_iee.pdf 
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Council of the Great City Schools, 2008 

Beating the odds: An analysis of student performance and achievement gaps on state 
assessments. Results from the 2006-2007 school year 



Focus 

Examines progress in student achievement among inner-city schools through spring 2007. 
Also measures achievement gaps between cities and states, minority and white students, 
and economically advantaged and disadvantaged students. Includes demographic and 
staffing data for the urban school districts studied. Part of a series of annual reports. 

Methodology 

Analyzed district- level achievement data for 66 urban school districts from 37 states and 
D.C. Data showed results of state math and reading tests used for NLCB and were collected 
from state Web sites, databases, and other sources. Trends covered the years with 
comparable data in each district between 2000-01 and 2006-07. 

Major Findings 

• Math gains. Mathematics achievement improved in urban schools. The percentage 
of graders in the Great City Schools scoring at or above the proficient level in 
math increased by 16 percentage points, from 47% in 2003 to 63% in 2007. The 
percentage proficient among urban graders similarly increased by 13 percentage 
points, from 42% to 55%. 

• Math achievement gaps. Gaps in math achievement in urban schools appeared to 
be narrowing. In 4*^^ and 8*^^ grade math, some progress was made in reducing racial- 
ethnic achievement gaps over the six years analyzed. 

• Reading gains. Reading achievement improved in urban schools. The percentage of 
4‘^ graders in the Great City Schools scoring at or above the proficient level in 
reading rose by 9 percentage points, from 51% in 2003 to 60% in 2007. The 
percentage proficient among urban 8‘^ graders similarly increased by 8 percentage 
points, from 43% to 51%. 

• Reading achievement gaps. Gaps in reading achievement in urban schools 
appeared to be narrowing. In 4‘^ and 8'^'^ grade reading, some progress was made in 
reducing racial-ethnic achievement gaps over the period analyzed. Urban school 
achievement is below state averages in reading. 

• Urban compared with state. Despite significant gains in performance, urban 
school achievement still stood below state averages in math and reading. 

• Urban context. The urban districts that belong to the Council of Great City Schools 
enrolled 15% of the nation's public school students, but a greater share of the 
nation's minority students. Students in urban schools are more likely to be African 
American, Hispanic, or Asian American; to come from low-income families; and to 
come from non-English speaking homes. 

Where to Obtain 

http://www.cacs.orq/publications/BT08 Revised.pdf 
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Economic Policy Institute, 2006 

"Proficiency for all"— An oxymoron 



Focus 

Examines whether NCLB's goal of 100% proficiency by 2014 is obtainable and if not, what a 

more reasonable goal would be. 

Methodology 

Investigated the varying meanings of proficiency according to NCLB and the National 

Assessment of Educational Progress and the distribution of student performance on NAEP 

and international exams. 

Major Findings 

• Unattainable goal. There is no date by which all (or even nearly all) students in 
any subgroup can achieve proficiency. By ignoring the inevitable and natural 
variation among individuals, NCLB is deeply flawed; no goal can simultaneously be 
challenging to and achievable by all students across the entire achievement 
distribution. Impossible gains would be required for all students to reach NAEP's 
challenging academic standard of proficiency. Even the world's top- performing 
countries are far from being able to meet a standard of "proficiency for all," as NAEP 
defines it. Remedial programs may contribute to higher achievement for cohorts 
already moving through the system but probably cannot succeed in realizing the 
goals of NCLB. 

• Gap-closing goal. Closing the achievement gap, which implies the elimination of 
variation between socioeconomic groups, is very difficult but worth striving for. 

• Shift in NAEP "proficient" definitions. Over time, NAEP has moved away from its 
original scale and norm- referenced reporting of results to criterion- referenced 
reporting, a move that has politicized standardized testing. The language and 
structure of NCLB law assumes "challenging" proficiency standards, signaling the 
intention that all students should be proficient according to NAEP's definition. 
However, NAEP's current definitions of proficiency are fraught with subjectivity. 
Making judgments of what students ought to be capable of, rather than basing 
judgments on observations of what actual students can achieve, yields results that 
the federal government itself acknowledges should be "interpreted with caution." 

• Lower standards not the answer. The problem with the 100% proficient goal 
can't be fixed by lowering NCLB's expectations, which would effectively return NCLB 
to the "minimum competency" accountability standard of the 1970s that the law was 
intended to reject. Even if the goal were lowered to meeting the equivalent of NAEP's 
"basic" performance level, many students would still fall short. 

• Alternative approach. The authors suggest a statistical procedure, inspired by 
business "benchmarking" and based on norm- referenced measures of academic 
achievement, that could be used to establish strenuous but realistic achievement 
goals. The repot describes a 19-year program that might bring a cohort of children 
from birth to maturity with high achievement. 

Where to Obtain 

http://www.epi.ora/webfeatures/viewpoints/rothstein 20061114.pdf 
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Education Trust, 2006 

Primary progress, secondary challenge: A state- by- state look at student achievement 

patterns 



Focus 

Examines patterns of student achievement from 2003 through 2005. 

Methodology: 

Analyzed state reading and math achievement data for 2003 through 2005 from the 
elementary, middle, and high school assessments used for NCLB accountability. Examined 
overall achievement patterns and progress in closing achievement gaps between groups of 
students. The study was limited to states with comparable data for the years analyzed, a 
total of 23 to 32 states, depending on grade level and subject. 

Major Findings 

• General trends. While student achievement in reading and math had risen in many 
states, much work lay ahead to ensure that all students met state standards. 
Progress in raising achievement and closing gaps was strongest in the elementary 
grades, and stronger in math than in reading. 

• Elementary level. Overall achievement gains were most consistent in the 
elementary grades, where math achievement increased in 29 of 32 states and 
reading achievement increased in 27 of 31 states. Elementary achievement declined 
in 1 state in math and in 3 states in reading. 

• Middle and high school levels. Middle and high school achievement improved 
somewhat, especially in math. In middle school math, 29 states improved overall 
achievement while 1 lost ground and 1 saw no change. Middle school trends were 
more mixed in reading, where overall achievement increased in 20 of the 31 states, 
declined in 6 states, and did not change in 5 states. The trend of more success in 
math than reading was seen in high school, as well. High school math results 
increased in 20 of 23 states and decreased in 2. High school reading results 
increased in 17 of 24 states and decreased in 5. 

• Gap trends. I n the elementary grades, 26 of 30 states narrowed the African 
American-white math gap. Twenty-four of 29 states narrowed the Latino-white 
reading gap. The majority of these states narrowed the gap by raising achievement 
for white students and simultaneously accelerating improvements for minority 
students. There was much less progress in narrowing gaps in the middle and high 
school grades, especially Latino-white gaps. Fewer states reported data to analyze 
gaps by income, but in states that did, high school trends were discouraging. In high 
school math, the gap between poor and non-poor students widened or stayed the 
same in 8 of the 12 states analyzed. In high school reading, this gap widened or 
stayed the same in 6 of 13 states. 

Where to Obtain 



http://www2.edtrust.org/NR/ rdonlvres/15B22876-20C8-47B8-9AF4- 
FAB148A225AC/0/PP5Creport.pdf 
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Education Trust, 2005 

Stalled in secondary: A look at student achievement since the No Child Left Behind Act 

Focus 

Examines student achievement on state assessments to determine whether the NCLB 
accountability provisions have helped to spur improvements. 

Methodology 

Examined achievement of students on state assessments from 2002 through 2004 in 29 
states at the elementary level, 28 states at the middle school level, and 23 states at the 
high school level. States with at least three years' worth of comparable, publicly available 
data were included in the analyses. 

Major Findings 

• Secondary progress lagging. After two full school years of implementing NCLB, 
states made progress in reading and math at the elementary grades, but results 
lagged in the middle grades and high schools, particularly when it came to narrowing 
achievement gaps. 

• Elementary trends. At the elementary level, 28 of the 29 states analyzed increased 
achievement from 2002 through 2005 in math, and 20 of 28 states raised 
achievement in reading. The majority of states reduced gaps between student 
groups in both math and reading at the elementary level. 

• Middle school trends. At the middle school level, 24 states improved overall 
performance in math, but only 16 of the 27 states examined raised scores in 
reading. Middle school reading achievement declined in eight states and did not 
change in three more. 

• High school trends. Fewer states raised overall performance in high schools. On 
math tests, 14 states made overall gains; 6 dropped in overall achievement, and 1 
saw no change. In reading, 11 states improved overall scores, while results declined 
in 6 states and remained flat in 3. 

• Gap trends. Many states are not achieving the goal of narrowing gaps between 
subgroups at the secondary level. In middle school reading, more states saw gaps 
narrow than widen, but in some cases gaps narrowed because achievement of white 
students went down. In middle school math, the Latino-white gap widened or stayed 
the same in more states than it narrowed in. The gap between poor and non- poor 
students narrowed in just nine states. States made the least progress in closing gaps 
at the high school level. 

• Consistency. In many states, progress was not consistent across grades. A handful 
of states lost ground for several groups across all three grades levels. 

Where to get the study online: 

http://www2.edtrust.org/NR/ rdonlvres/77670E50-188F-4AA8-8729- 
555115389E18/0/5talledl n5econdarv.pdf 
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Human Resources Research Organization (HumRRO), 2008 

Are advanced students advancing? Examining achievement trends beyond proficiency 

Focus 

Examines changes in the percentage of students scoring at the "advanced" or top 
achievement level on the state tests used for NCLB. Also looks at the magnitude of 
achievement gaps for subgroups of students at the highest achievement levels. 

Methodology 

Analyzed test data provided and verified by all 50 states on students performing at the top 
achievement level at the elementary and middle school grades from 2002 to 2006. Analyses 
of trends were limited to states with comparable years of data. Data were originally 
collected for the achievement studies of the Center on Education Policy. 

Major Findings 

• State differences in top group. There were wide disparities across states in the 
percentages of students classified in the top achievement level, ranging from a 
minimum of 2% to a maximum of 51% to 62%, depending on the subject and grade 
assessed. Average percentages of students classified in the top achievement level 
ranged from 17% (middle school math) to 23% (middle school reading). Interpreting 
these differences is complicated, however, because state definitions of top-level 
(advanced) performance vary widely. 

• Gains for top group. I n the majority of states, the average annual percentage of 
students classified in the top achievement level increased across all grade and 
subject combinations. Still, several states saw declines in the percentages of top- 
performing students in each subject/grade combination analyzed. Elementary grades 
tended to have slightly greater increases than middle grades, and more states 
showed gains in math than in reading. About one- fourth of the states experienced 
declines in the percentages of top- performing students in both elementary and 
middle school reading. 

• Widening gaps. Across grade levels, subjects, and subgroups, achievement gaps at 
the top level generally did not narrow. This was particularly true in math, where 
about three-quarters of the states with comparable data saw gaps widen. And in 
reading, more than half the states with comparable data saw gaps widen. The 
average changes in gaps at the top performance level varied across states. 

Where to Obtain 

http://www.humrro.orq/corpsite/sites/default/files/downloads/biblio/Are%20Advanced%20 
5tudents% 20Advancinq-- 

Examininq% 20Achievement% 20Trends% 20Bevond% 20Proficiencv. pdf 
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Northwest Evaluation Association, 2005 

The impact of the No Child Left Behind Act on student achievement and growth: 2005 

edition 



Focus 

Investigates the impact of NCLB on student achievement and growth by comparing 
achievement trends before (school year 2001-02) and after (school year 2003-04) the 
implementation of NCLB. Also examines the impact of NCLB on the performance and 
growth of students in various subgroups. 

Methodology 

Used data from the Growth Research Database of the Northwest Evaluation Association 
(NWEA), which allows student achievement and growth to be compared on a common, 
stable scale. Examined performance on the NWEA tests in reading for more than 320,000 
students in grades 3 through 8 in 200-plus school districts in 23 states, and in mathematics 
for more than 334,000 students in grades 3 through 8 in 200-plus school districts in 22 
states. 

Major Findings 

• Overall gains in reading and math. Reading and mathematics scores improved in 
the first two years after NCLB was implemented (between 2001-02 and 2003-04). 

• Less growth for all racial-ethnic groups. The rate of student growth declined in 
the first two years after NCLB was implemented. Slight decreases in growth were 
found for every major racial-ethnic group. 

• Greater changes in math. The overall gains in test scores— as well as the declines 
in the rate of growth— were greater in math than in reading. 

• Higher achievement in grades with state tests. Students in grades tested in 
state assessment programs had higher achievement and growth on the NWEA tests 
(an independent measure) than students in grades not included in state 
assessments. The presence of a state test that measures, monitors, and reports 
student achievement appeared to provide some impetus to improved learning. 

• Shrinking achievement gaps. Gains for African American, Native American, and 
Hispanic students were large enough to cause a modest narrowing of the 
achievement gaps between these subgroups and white and Asian students. 

• Less growth for Hispanic students. In every grade and subject analyzed, growth 
of Hispanic students tended to be lower than that of Anglo students with exactly the 
same initial score. 

Where to Obtain 

http://www.nwea.orq/research/nclbstudv.asp 
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Policy Analysis for California Education, University of California- Berkeley, 
University of California- Davis, and Stanford University, 2007 

Gauging growth: How to judge No Child Left Behind? 



Focus 

Seeks to judge the effects of NCLB on achievement by examining long-term national trends 
on the National Assessment of Educational Progress, the share of students deemed 
proficient under state versus NAEP definitions, and comparisons of trends on state tests and 
NAEP. 

Methodology 

Analyzed data from NAEP and state assessments in 12 diverse states for the period from 
1992 to 2006, with particular focus on the performance of graders. 

Major Findings 

• I mportance of NAEP trends. To answer the question of whether NCLB is working, 
one should not rely on state testing programs and the jagged trend lines that stem 
from their results. Instead, it is important to focus on historical patterns informed by 
NAEP. State tests exaggerate the percentage of 4'^'^ graders deemed proficient and 
above compared with NAEP results. 

• Pre- and post-NCLB trends. In the 12 states analyzed, earlier test score growth 
largely faded after enactment of NCLB in 2002. Gains in math achievement persisted 
in the post-NCLB period, albeit at a slower rate of growth. 

• Different proficiency definitions. Performance in many states appeared to 
continue climbing after NCLB. But the bar defining proficiency has been set much 
lower in most states than the NAEP definition of proficiency, and the disparity 
between state and NAEP results has grown since 2001. 

• Progress in narrowing gaps. Progress seen in the 1990s in narrowing 
achievement gaps has largely disappeared in the post-NCLB era. 

Where to Obtain 



http: //WWW. aera.net/uploadedFiles/ Publications/I ournals/ Educational Researcher/3605/07E 
DR07 268-278.pdf 
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Policy Analysis for California Education, University of California- Berkeley, 
University of California- Davis, and Stanford University, 2006 

Is the No Child Left Behind Act working? The reliability of how states track achievement 

Focus 

Examines whether state testing systems provide an accurate and consistent indicator of 
students' proficiency and whether the federal rules and resources are prompting 
achievement gains. 

Methodology 

Compared trends from 1992 through 2005 in state test results and results from the National 
Assesment of Educational Progress for 12 diverse states. Also analyzed achievement 
changes for the three years after NCLB was enacted in 2002. 

Major Findings 

• Reading. Across 12 states there was a small improvement in the percentage of 
children achieving proficiency in reading, based on NAEP results between 1992 and 
2005. But state tests estimated much higher shares of students reaching proficiency 
than the NAEP results show. 

• Math. Children made greater progress in math proficiency than in reading over the 
13-year period analyzed, but state test results exaggerated the annual rate of 
improvement compared with NAEP results. 

• Disparities between state and NAEP results. Disparities between state and NAEP 
estimates of proficiency are not new. States have long claimed that a much higher 
share of students are proficient relative to NAEP results, even before NCLB created 
the incentive for states to set a low bar. 

• Post-NCLB trends. During the three school years after NCLB was enacted in 2002, 
some states maintained their apparent momentum in raising the percentage of 4*^^ 
graders scoring proficient in math, as gauged by state and NAEP exams, while 
reading performance leveled off or slipped in several states. Two states with weak 
accountability systems prior to NCLB (Arkansas and Nebraska) experienced gains in 
math proficiency after enactment of NCLB but not in reading. 

Where to Obtain 

http://pace.berkelev.edu/2006/06/01/is-the-no-child-left-behind-act-workinq-the-reliabilitv- 
of- how- states- track- achievement/ 
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RAND Education, 2005 

Achieving state and national literacy goals: A long uphill road 



Focus 

Investigates the extent to which adolescents (students in grades 4 through 12) are meeting 
state and national literacy goals as measured by state assessments and the National 
Assessment of Educational Progress, and whether results from these two types of 
assessments are consistent with one another. 

Methodology 

Collected assessment data in English language arts from all states, mostly from testing year 
2003. Examined NAEP results for grades 4 and 8 from 2003 in reading and 2002 in writing. 

Major Findings 

• General findings. In several states, fewer than half of the students met the state 
proficiency standards for literacy set for NCLB, and in no state did even half the 
students meet the NAEP national literacy standard of proficiency. Moreover, the pass 
rates on state assessments varied significantly from state to state. 

• State reading results. The pass rates on the elementary school (4'^^ or 5‘^-grade) 
state assessments differed widely across states, ranging from 28% to 87%. In seven 
states, less than half of the students passed at the elementary level. The pass rates 
on the middle school state assessments ranged from 21% to 88% percent. Three 
states had pass rates of less than 30%. In 12 states, less than half of the students 
passed the reading assessment. 

• NAEP reading results. Grade 4 NAEP proficiency rates on the 2003 state- by- state 
NAEP reading assessment ranged from 10% to 43%, with an average of 30%. In 
only three states (Massachusetts, New Hampshire, and Connecticut) did the 
proportion of students scoring at the proficient level reach 40% or above. Overall, 
between 10% and 43% of 8‘^ graders scored at the proficient level on the 2003 NAEP 
reading assessment. The average proficiency rate of 8‘^ graders was 32%. 

• Writing tests. Pass rates on the state and the NAEP writing assessments tended to 
be somewhat lower than on the reading assessments. 

• Achievement gaps. The NAEP and state assessments showed large and surprisingly 
similar achievement gaps between subgroups of students disaggregated by 

race/ ethnicity and poverty status. Students with limited English proficiency and 
students with disabilities trailed well behind their peers. 

Where to Obtain 

http://www.rand.orq/pubs/technical reports/ 2 005/ RAND TR180-l.pdf 
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Thomas B. Fordham I nstitute, 2008 

High-achieving students in the era of NCLB 



Focus 

Examines the NAEP performance of high-achieving students during the 1990s and since 
2000. Analyzes teacher views of how schools are serving high-achieving students. 

Methodology 

Analyzed national and state NAEP performance data and demographic data from restricted- 
use NAEP student files. Conducted a random, nationally representative survey in winter- 
spring 2008 of the attitudes of 900 teachers in grades 3-12 toward the education of 
academically talented students. 

Major Findings 

• Languid performance for high achievers. Since 2000, children at the 10‘^ 
percentile of achievement (the bottom 10% of students) have made rapid gains on 
NAEP 4th-grade reading and math tests and on 8‘^-grade math tests, but those at 
the 90*^^ percentile (the top 10%) have made minimal gains. 

• Not just an NCLB phenomenon. The pattern of big gains for low achievers and 
lesser ones for high achievers is associated with the introduction of high-stakes 
accountability in general, not just NCLB. NAEP data from the 1990s show that states 
that adopted testing and accountability regimes before NCLB saw similar patterns 
before NCLB. 

• Struggling, not advanced, students a top priority. Asked about the needs of 
struggling students, 60% of teachers said they were a "top priority" at their school. 
Asked about the needs of "academically advanced" students, only 23% of teachers 
said they were a top priority. (They could give multiple answers to this question.) 

• More teacher attention to low-achieving students. Asked "who is most likely to 
get one-on-one attention from teachers?" 81% of teacher named struggling students 
while only 5% named advanced students. 

• Beiief that all students deserve equal attention. Teachers were asked the 
following: "For the public schools to help the U.S. live up to its ideals of justice and 
equality, do you think it's more important that they (a) focus on raising the 
achievement of disadvantaged students who are struggling academically, or (b) 
focus equally on all students, regardless of their backgrounds or achievement 
levels?" Only 11% chose the former, while 86% chose the latter. 

• Teachers experience. Low-income, black, and Hispanic high achievers (on the 
2005 8‘^-grade math NAEP) were more likely than low achievers to be taught by 
experienced teachers. These groups of high achievers were also as likely as other 
high-achieving students to have teachers who had majored or minored in math. 

Where to Obtain 

http://www.edexcellence.net/doc/20080618 high achievers.pdf 
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U. S. Department of Education, Institute for Education Sciences, 2008 

Comparison between NAEP and state mathematics assessment results, 2003: 
Volume 1, research and development report 



Focus 

Considers whether NAEP results and state assessment results are comparable. 

Methodology 

Analyzed 2003 data from NAEP and state tests for 48 states and D.C. 

Major Findings 

• State vs. NAEP standards. The median standard for proficiency on states' primary 
math assessment fell between the NAEP basic and proficient levels in grades 4 and 8. 

• Variation. Standards varied greatly in difficulty across states, as reflected in their 
NAEP equivalents. There was more variation among states in the placement of state 
standards on the NAEP scale than there was in average NAEP performance. 

• Link between standards and performance. States with high standards tended to 
see few students meet their standards, while states with low standards tended to see 
more students meet standards. There is no evidence that setting a higher standard 
correlated with higher NAEP performance. 

• State vs. NAEP trends. There were significant differences in trends between the 
two assessments in 14 of 24 states for grade 4 and in 11 of 22 states for grade 8. 

• Correlations across schools. Correlations across schools between NAEP and state 
assessment results in math were greater than 0.7 in 41 of 46 states for grade 8 and 
in 30 of 49 states for grade 4. (A correlation of at least 0.7 is important for 
confidence in the linkages.) But several factors depressed the correlations. 

• Some gains larger on NAEP. Across states, NAEP achievement gains from 2000 to 
2003 were significantly larger than state assessment gains in grade 4 but not grade 
8. Across states, there was a positive correlation in gains between the two tests. 

• Gap comparisons. In 34 of 70 gap comparisons at grade 4, and in 17 of 62 gap 
comparisons at grade 8, NAEP showed significantly larger achievement gaps than the 
state assessment did. In only two gap comparisons (both at grade 8), the state 
assessment found a significantly larger gap. The tendency for NAEP to show larger 
gaps in math than state tests did was equally strong for the black-white and 
Hispanic-white gaps, but was slightly weaker for poverty gap comparisons. 

Where to Obtain 

http://nces.ed.qov/pubs2008/2008475 l.pdf 



CEP, Compendium of Major NCLB Studies 



Achievement & Related Issues 



Page 20 



U.S. Department of Education, Policy and Program Studies Service, 2007 

National Assessment of Title I final report: Vol. I, implementation 



Focus 

Comprehensive study that describes the progress of states, districts, and schools through 
school year 2004-05 in implementing key provisions of Title I. Summarized below are the 
study's findings about achievement. 

Methodology 

Drew on data from a set of implementation studies by the U.S. Department of Education. 
Data for these studies came from surveys conducted in a nationally representative sample 
of school districts, other state and local surveys, state performance reports, and the 
National Assessment of Educational Progress. 

Major Findings 

• Overall achievement. For both state assessment and NAEP results, recent 
achievement trends through 2004 or 2005 are positive overall and for key 
subgroups, particularly in mathematics and at the elementary level. In states that 
had three-year trend data available from 2002-03 to 2004-05, the percentage of 
students achieving at or above the state's proficient level rose for most student 
subgroups in a majority of the states. 

• Pace toward 100% . Based on trend data for 36 states, most states would not meet 
the goal of 100% proficient by 2013-14 unless achievement increased at a faster 
rate. 

• NAEP trends. Recent NAEP trends showed gains for 4*^^ grade students in reading, 
mathematics, and science, overall and for minority students and students in high- 
poverty schools. Trends for middle and high school students were mixed; neither 
nor 12‘^ grade students made gains in reading or science achievement. 

• Achievement gaps. State assessments and NAEP both provided some indications 
that achievement gaps between disadvantaged students and other students may be 
narrowing. For example, state assessments showed a reduction in the gap between 
low- income students and all students in most states. On the long-term trend NAEP, 
achievement gains for black and Flispanic students substantially outpaced gains by 
white students, resulting in significant declines in the black-white and Flispanic-white 
achievement gaps, but recent changes in gaps often were not statistically significant. 

• Graduation rates. The averaged freshman graduation rate was fairly steady from 
1996 to 2004. The mean graduation rate in 2004 (75%) was slightly higher than in 
1996 (73%). Flowever, these longitudinal data may not be strictly comparable 
because of changes in reporting over time. 

Where to Obtain 

http://ies.ed.qov/ncee/pdf/20084014_rev.pdf 
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Center on Education Policy, 2009 

Lessons from the classroom level about federal and state accountability in Rhode Island and 

I llinois 



Focus 

Examines the impact of NCLB and related state accountability policies on curriculum, 
instruction, and student achievement in several schools and classrooms in Rhode Island and 
I llinois. 

Methodology 

Conducted case studies of 12 schools in Rhode Island and Illinois in the winter and spring of 
2007-08. In each state, interviewed dozens of district- and school-level administrators, 
instructional specialists, and instructional coaches, and held focus groups with scores of 
teachers, students, and parents. Also conducted formal "time-sampling" observations in 
several classrooms within these schools to document the time spent on various types of 
instructional practices and interactions. 

Major Findings 

Common findings from both Rhode Island and Illinois: 

• Time for test preparation. In both states, many administrators and teachers 
reported using various forms of test preparation to familiarize students with the 
content and format of the tests used for NCLB accountability. 

• Greater emphasis on teacher-led instruction. I n most of the classrooms that 
were the subject of time-sampling observations, teachers in both states spent a 
considerable portion of class time asking "closed questions"— those with just one or a 
few correct answers. Other types of more independent learning activities, such as 
students working in learning centers, were used somewhat less frequently. 

• More time on tested subjects. Study participants reported focusing more 
instructional time on the tested subjects of English language arts and mathematics at 
the expense of other subject areas. 

• Greater use of data. In both states, administrators and teachers interviewed 
reported making greater use of test data to reach decisions about curriculum, 
instruction, teacher professional development, and other areas. 

Where to Obtain 

www.cep-dc.orq 
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Center on Education Policy, 2008 

Instructional time in elementary schools: A closer look at changes for specific subjects* 

Focus 

Takes a closer look at the magnitude of the shifts in instructional time that school districts 

made between 2001-02 (the year NCLB was enacted) and 2006-07. 

Methodology 

Based on a nationally representative survey of 349 responding school districts conducted 

during school year 2006-07. This analysis focuses on districts that reported in this survey 

that they had increased instructional time for English language arts (ELA) and mathematics 

Major Findings 

• Magnitude of changes. The shifts in instructional time toward ELA and math and 
away from other subjects were relatively large in a majority of school districts that 
made these types of increases and decreases. Districts that increased instructional 
time for ELA and/or math did so by 43%, on average. Districts that also reduced 
instructional time in other subjects reported total reductions of 32%, on average. 

• Minutes per week. Eight out of ten districts that reported increasing time for ELA 
did so by at least 75 minutes per week, and more than half (54%) did so by 150 
minutes or more per week. Among districts that reported adding time for math, 63% 
added at least 75 minutes per week, and 19% added 150 minutes or more per week. 

• Cuts in other subjects. Most districts that increased time for ELA or math also 
reported substantial cuts in time for other subjects or activities, including social 
studies, science, art and music, physical education, recess, or lunch. Seventy-two 
percent of the districts with increases in ELA or math reported that they had reduced 
time by a total of at least 75 minutes per week for one or more of these other 
subjects. For example, more than half (53%) of these districts cut instructional time 
by at least 75 minutes per week in social studies, and the same percentage (53%) 
cut time by at least 75 minutes per week in science. 

Where to Obtain 

www.cep-dc.org 
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Center on Education Policy, 2007 

Choices, changes, and challenges: Curriculum and instruction in the NCLB era* 



Focus 

Examines changes in curriculum and instructional time made by school districts between 

2001-02 (the year NCLB was enacted) and 2006-07. 

Methodology 

Based on a nationally representative survey of 349 school districts conducted during 2006- 

07 and interviews with district- and school- level leaders and staff in 13 districts. 

Major Findings 

• More time for tested subjects. About 62% of districts reported they had increased 
time for English language arts and/or math in elementary schools since 2001-02, 
and more than 20% reported increasing time for these subjects in middle school. 
Since 2001-02, those districts with increases reported a 47% increase in minutes per 
week for ELA, a 37% increase in math, and a 43% increase for both subjects 
combined. 

• Less time for other subjects. To accommodate the increased time for ELA and 
math, 44% of districts reported cutting time from one or more other subjects or 
activities (social studies, science, art and music, physical education, lunch and/or 
recess) at the elementary level. 

• Schools in improvement. Shifts in instructional time are more common in districts 
with at least one school identified for improvement than in with districts with no 
schools in improvement. 

• More emphasis on tested content and skills. Since 2001-02, most districts have 
changed their ELA and math curricula to put greater emphasis on the content and 
skills covered on state tests used for NCLB. In elementary level reading, 84% of 
districts reported that they had changed their curriculum "somewhat" or "to a great 
extent" to emphasize tested content. In middle school ELA, 79% reported making 
this change, and in high school ELA, 76%. In math, 81% of districts reported that 
they had changed their elementary and middle level curricula to emphasize tested 
content and skills, and 78% reported having done so at the high school level. 

Where to Obtain 

www.cep-dc.orq 






For a more detailed analysis, see the summary for the CEP report. Instructional Time in Elementary Schools: A 
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Center on Education Policy, 2006 

From the capital to the classroom: Year 4 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2004-05 and 2005-06. Summarized below 
are the study's findings about impact on curriculum and instruction. 

Methodology 

Collected data through a survey of all 50 states, a nationally representative survey of 299 
school districts, case studies of 38 geographically diverse districts and 42 schools, three 
national forums, and six special analyses of critical issues in implementing NCLB. 

Major Findings 

• Curriculum alignment and data use. Case study data indicated that 
administrators and teachers had made a concerted effort to align curriculum and 
instruction with state academic standards and assessments. Principals and teachers 
also reported they were making better use of test data to adjust teaching to address 
students' individual and group needs. 

• Required time for reading and math. Of the districts surveyed, 60% had policies 
that required teachers to devote a specific amount of time to reading in elementary 
schools, and 50% had policies that required a specific amount of time for math. 
Nearly all (97%) of the highest- poverty districts had policies specifying the amount 
of time to be spent on reading, compared with 55% of the lowest- poverty districts. 

• Narrower curriculum. Seventy-one percent of the school districts surveyed 
reported in 2005-06 that they had reduced elementary school instructional time in at 
least one other subject to make more time for reading and mathematics. In some 
case study districts, struggling students received double periods of reading or math 
or both— sometimes missing certain subjects altogether. Some officials in case study 
districts viewed this extra time for reading and math as necessary to help low- 
achieving students catch up. Others felt that this practice shortchanged students 
from learning important subjects, squelched creativity in teaching and learning, or 
diminished activities that might keep children interested in school. 

• Greater direction about teaching. Many case study districts had become more 
prescriptive about what and how teachers were supposed to teach. Some districts 
encouraged teachers to follow pacing guides that outlined the material to be covered 
at different points in the school year, while others hired instructional coaches to 
observe teachers teaching, demonstrate model lessons, and give teachers feedback 
on ways to improve. 

Where to Obtain 

www.cep-dc.orq 
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Center on Education Policy, 2005 

From the capital to the classroom: Year 3 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2003-04 and 2004-05. Summarized below 
are the study's findings about impact on curriculum and instruction. 

Methodology 

Collected data through a survey of 49 states, a nationally representative survey of 314 
school districts, case studies of 36 geographically diverse districts and 37 schools, three 
national forums, and four special analyses of critical issues in implementing NCLB. 

Major Findings 

• Instructional time. About one-fifth of the districts surveyed reported in 2004 that 
they had changed their policies as a result of NCLB to require more instructional time 
in reading and mathematics. But 27% of districts said they had reduced the time 
devoted to social studies somewhat or to a great extent, 22% reported reducing time 
for science, and 20% reported reducing time for art and music. 

Where to Obtain 

www.cep-dc.org 
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Government Accountability Office, 2009 

Access to arts education: Inclusion of additional questions in Education's planned research 
would help explain why instruction time has decreased for some students 



Focus 

Assesses whether the amount of instructional time for arts education changed since 
enactment of NCLB, and, if so, whether certain groups have been affected more than 
others. Also examines whether state requirements and funding for arts education have 
changed and what school officials are doing to provide arts education since NCLB. 

Methodology 

Analyzed data from the U.S. Department of Education, surveyed 50 state arts officials, 
interviewed officials in 8 school districts and 19 schools, and reviewed existing research. 

Major Findings 

• No change in most classrooms. About 90% of elementary school teachers 
reported that instructional time for arts education stayed the same between school 
years 2004-and 2006-07, according to data from an ED survey. This percentage was 
similarly high across schools with a range of characteristics, including the schools' 
percentages of low-income, minority, or English language learner students or the 
schools' improvement state under NCLB. 

• Reductions and increases in some classrooms. About 7% of teachers reported a 
decrease in instructional time for arts education, while about 4% reported an 
increase. Teachers at schools identified for NCLB improvement or with higher low- 
income or minority enrollments reported larger average reductions in time for the 
arts. Among the 7% of teachers reporting decreases, those at schools with high 
percentages of low- income students reported greater reductions in time for arts 
education (an average decrease of 49 minutes per week) than those at schools with 
smaller low-income enrollments (an average decrease of 31 minutes per week). The 
ED survey did not include questions about the reasons for these changes. 

• Little change in state requirements but some changes in state funding. While 
basic state requirements for arts education in schools have remained unchanged in 
most states, state funding levels for arts education increased in some states and 
decreased in others, according to GAO's survey of state arts officials. Arts education 
officials attributed the funding changes to state budget changes more often than 
they did to NCLB. 

• School strategies and competing demands for time. School principals have 
used several strategies to provide arts education, including seeking funding and 
collaborative arrangements in the arts community. Some principals reported that 
they struggled with competing demands on instructional time due to state or district 
actions taken to meet NCLB proficiency standards. 

Where to Obtain 

http://www.qao.qov/new.items/d09286.pdf 
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The Urban Institute and RAND, 2008 

Performance- based accountability policies: Implications for school and classroom practices 

Focus 

Describes school and classroom- level responses to performance- based accountability, 
especially how curriculum and instructional practices are aligned with standards. Examines 
the costs and benefits of performance- based accountability and its intermediate outcomes. 

Methodology 

Reviewed evidence from dozens of studies of performance- based accountability, including 
survey results from teachers and administrators in states that instituted this type of 
accountability; national survey data from teachers and administrators; student performance 
information by subject; and qualitative findings and insights from small case studies. 

Major Findings 

• Impact on instruction. In performance- based accountability systems, instructional 
time for tested content has increased, sometimes at the expense of other subjects, 
which has resulted in curriculum narrowing. For example, time spent on math 
instruction in elementary schools increased by 40% between school years 1999-2000 
and 2003-04, as state and federal accountability plans were implemented. Some 
evidence suggests that the magnitude of the shifts among subjects varies according 
to school achievement, with greater shifts in lower- performing schools. Decisions of 
schools and districts, as well as teachers, play a role in how teachers allocate time 
and what content is covered. Test format also affects the skills and content 
emphasized in instruction. Some evidence suggests that state tests have led 
teachers to teach in ways that run counter to their ideas of good practice. 

• Behaviors intended to increase test scores. Teachers reported spending a 
significant amount of time developing students' test-taking skills, a practice often 
reinforced by administrators. However, the distinction is not necessarily clear 
between teaching that improves students' knowledge and skills and teaching that 
artificially inflates test scores. There are indicators that outright cheating is 
occurring. In several studies, teachers have reported focusing more attention on 
students who were close to meeting proficiency targets. Other studies provided 
evidence that some students' test results were excluded from accountability or that 
students were assigned to special education to influence testing outcomes. 

• Benefits. Accountability policies produce information that can inform instruction and 
make teaching more efficient and effective. Teachers have reported responding to 
accountability by focusing more strongly on achievement than previously, working 
harder, and seeking to improve their own practices in tested subjects. The ways 
teachers use data are likely to be affected by such factors as school capacity and 
resources related to data, opportunities for interactions with colleagues, and 
timeliness and accessibility of data 

• Infrastructure. Under some conditions, accountability systems affect resource 
allocation and staffing patterns in ways likely to support instructional practice. 

Where to Obtain 

http://www.urban.ora/UploadedPDF/411779_accountabilitv_policies.pdf 
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U.S. Department of Education, Policy and Program Studies Service, 2007 

National Assessment of Title I: Final report, vol. I 



Focus 

Comprehensive study that describes the progress of states, districts, and schools through 
school year 2004-05 in implementing key provisions of Title I. Summarized below are the 
study's findings about curriculum and instruction. 

Methodology 

Drew on data from a set of implementation studies by the U.S. Department of Education. 
Data for these studies came from surveys conducted in a nationally representative sample 
of school districts, other state and local surveys, state performance reports, and the 
National Assessment of Educational Progress. 

Major Findings 

• Increases in instructional time within schools in improvement. Nearly one- 
third (30%) of elementary schools identified for improvement reported increasing the 
amount of instructional time in reading by more than 30 minutes per day in 2004-05, 
and 17% reported a similar increase in instructional time for mathematics. Schools 
not identified for improvement less frequently reported these types of increases. 
Identified secondary schools also more commonly reported increasing instructional 
time for low-achieving students in reading (55% compared with 36% of non- 
identified schools). 

• Extended learning time. Almost three-fourths of all schools offered extended-time 
instructional programs, and the percentage of students served through after-school 
programs doubled from 1997-98 to 2004-05 (from 5% to 10%). In schools that 
implemented after-school programs, the programs provided an additional 134 hours 
of instruction annually, on average, or about a 12% increase in instructional time for 
participating students. 

Where to Obtain 

http://ies.ed.qov/ncee/pdf/20084014 rev. pdf 
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Arizona State University, 2005 

No Child Left Behind: Where does the money go? 



Focus 

Examines the various opportunities under NCLB for federal dollars to flow eventually to 
private, for-profit companies. 

Methodology 

Analyzed NCLB legislative requirements and roles of private companies; analyzed data on 
estimated or potential costs of various NCLB mandates. 

Major Findings 

• A large sum of money under NCLB has flowed through the states and districts into 
the coffers of private companies and individuals. 

• A look at the consequences of failing to meet NCLB requirements reveals the 
"stunning double standard of the feet-to-the-fire treatment of public schools 
contrasted with the lax treatment of private corporations that provide materials or 
services the law requires the schools to use." 

• Under NCLB, several billions of taxpayer dollars are likely to flow each year to private 
sources for the following opportunities (and estimated amounts): 

> Test development, scoring and reporting: $2.29 billion per year by 2006 
(Eduventures estimate) 

> Reading First curriculum adoption: potential annual amount of $1.1 billion, actual 
amount unknown 

> Curriculum adoption for other curriculum areas and grades: potential and actual 
amounts unknown 

> School choice and supplemental educational services: potential annual amount of 
$2 billion, actual unknown 

> Restructuring of low-performing schools: potential and actual amounts unknown 

> Contracting for services to monitor cheating on tests: unknown. 

> Contracting to develop student-tracking databases: unknown. 

> Professional development: unknown. 

• No real process of accountability was in place to monitor where the money was spent 
or how effectively it was spent. History shows that under such conditions money is 
wasted and fraudulent expenditures are likely. 

Where to Obtain 

http://epsl.asu.edu/epru/documents/EPSL-0506-114-EPRU.pdf 
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Center on Education Policy, 2008 

Title I funds— Who's gaining, who's losing: School year 2008-09 update 



Focus 

Analyzes changes in allocations to states and school districts for school year 2008-09 under 

Title I, Part A, of ESEA, as amended by NCLB. 

Methodology 

Analyzed data on Title I allocations to school districts for use in 2008-09, released by the 

U.S. Department of Education in J une 2008. 

Major Findings 

• Funds for school improvement. For school year 2008-09, roughly $1 billion was 
available to assist schools identified for improvement under NCLB, more than double 
the amount available for school improvement for school year 2007-08. This increase 
was the result of 1) growth in overall appropriations for Title I, which enabled states 
to reserve more funds for the 4% state set-aside for school improvement, and 2) an 
increase in a special Congressional appropriation for school improvement. 

• Fluctuations. Title I allocations to states and districts continued to fluctuate 
substantially as a result of year-to-year changes in census estimates of the numbers 
of low- income children. Seventeen states had yearly increases of more than 10% in 
their child poverty estimates. One state that had an increase of more than 30% in its 
child poverty estimate for the previous year had a 9% decrease for 2008-09. 

• Katrina funds. Federal assistance to school districts affected by Flurricane Katrina 
had the unintended consequence of reducing Title I funds to Louisiana and, to a 
lesser degree, Mississippi. 

• Funds to high-poverty districts. Changes made in the Title I formula in 2002 that 
were intended to target significantly more funds to the highest- poverty districts have 
provided only somewhat more funds to these districts. The amount of increase is 
limited because approximately 50% of the funds are still distributed under the Basic 
and Concentration Grant formulas in the law. 

Where to Obtain 

www.cep-dc.orq 
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Center on Education Policy, 2007 

Educational architects: Do state education agencies have the tools necessary to implement 

NCLB? 



Focus 

Analyzes the funding, staffing, and technological capacity of state education agencies to 
carry out the accountability demands and other requirements of NCLB. 

Methodology 

Conducted a survey of 50 states and interviews with 15 high-ranking state education 
officials from 11 states in 2006-07. Analyzed state policy documents and other research on 
state capacity issues. 

Major Findings 

• Capacity to implement NCLB. NCLB significantly increased the responsibilities of 
state education agencies, creating capacity challenges in many states. States with 
high percentages (26% or more) of schools that missed AYP targets reported having 
more capacity challenges in implementing NCLB than states with lower percentages. 

• I nsufficient staff. I nsufficient staffing was the capacity challenge most often 
mentioned by states in 2006-07. This particularly affected states' ability to provide 
technical assistance to districts with schools in improvement, oversee the activities of 
these districts, and monitor supplemental educational services. States with 
populations under 1 million reported being more significantly affected than other 
states by insufficient staff to assist schools and districts identified for improvement. 

• Federal funding. Inadequate federal funding was a challenge to most states; 32 
states reported that Title I funds were insufficient to improve achievement in schools 
identified for improvement. Inadequate federal funding challenged the capacity to 
monitor SES providers to a great extent in 25 states, and challenged the capacity to 
provide technical assistance to districts to a great or moderate extent in 41 states. 

• Technical assistance. Only 11 states reported that they were able to provide 
technical assistance to a great extent to districts with schools in NCLB improvement. 
Insufficient numbers of staff, lack of in-house expertise, and inadequate federal and 
state funding were the major impediments to implementing this requirement. 

• Data systems. States differed in their capacity to create and maintain data 
systems, but officials in most states agreed that NCLB has put additional strain on 
the technological capacity of their agency. 

• ED guidance. Most states noted that guidance about NCLB implementation from the 
U.S. Department of Education was not very helpful. The ED guidance rated most 
helpful pertained to Reading First; the least helpful guidance was on monitoring SES. 

Where to Obtain 

www.cep-dc.org 
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Center on Education Policy, 2007 

Title I funds— Who's gaining, who's losing: School year 2007-08 update 



Focus 

Analyzes changes in allocations to states and school districts for school year 2007-08 under 

Title I, Part A, of ESEA, as amended by NCLB. 

Methodology 

Analyzed data on Title I allocations to school districts for use in 2007-08, released by the 

U.S. Department of Education. 

Major Findings 

• School improvement funds. For school year 2007-08, funds for school 
improvement under NCLB were distributed erratically among the states. Because of 
the law's "hold harmless" provision, three states were unable to reserve any funds 
for the 4% state set-aside for school improvement, and 29 could not reserve the full 
4%. The additional, separate appropriation of $125 million for school improvement 
was expected to help somewhat. 

• I mpact of census updates. The amount of Title I , Part A funding for some states 
and school districts has fluctuated from year to year due to annual updating of 
census estimates of the number of children in poverty. Because states' relative 
shares of the total number of low-income children shifted, some states received 
double-digit increases in Title I funding for 2007-08, while others lost substantial 
funds. These shifts in turn affected how much school districts within a state received. 

Where to Obtain 

www.cep-dc.orq 
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Center on Education Policy, 2006 

From the capital to the classroom: Year 4 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2004-05 and 2005-06. Summarized below 
are the study's findings about funding. 

Methodology 

Collected data through a survey of all 50 states, a nationally representative survey of 299 
school districts, case studies of 38 geographically diverse districts and 42 schools, three 
national forums, and six special analyses of critical issues in implementing NCLB. 

Major Findings 

• Greater burden, inadequate funding. States and districts lacked both the funding 
and the staff capacity to carry out all of the demands of NCLB. Some 80% of the 
school districts surveyed noted that they had costs for NCLB that were not covered 
by federal funds. Thirty-three states reported that federal funds were inadequate to 
assist all schools identified for improvement, and less than half of school districts 
said they had enough money to assist identified schools at least somewhat. In 
addition, 36 states reported they did not have enough staff to implement NCLB— a 
major concern because state agencies were the source that school districts most 
often turned to for help in implementing NCLB. 

Where to Obtain 

www.cep-dc.orq 
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Center on Education Policy, 2005 

From the capital to the classroom: Year 3 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2003-04 and 2004-05. Summarized below 
are the study's findings about impact on curriculum and instruction. 

Methodology 

Collected data through a survey of 49 states, a nationally representative survey of 314 
school districts, case studies of 36 geographically diverse districts and 37 schools, three 
national forums, and four special analyses of critical issues in implementing NCLB. 

Major Findings 

• Lack of capacity. A great majority of states— 45 states— reported that limited staff 
size posed a serious or moderate challenge in implementing NCLB, and 31 said that 
limited staff expertise presented a serious or moderate challenge. Furthermore, 42 
states said that providing assistance to all schools identified for improvement was a 
serious or moderate challenge. 

• Insufficient funding. Most states and districts indicated that federal funds were not 
sufficient to carry out all aspects of NCLB. Only 11 states felt NCLB allocations were 
adequate for them to provide technical assistance to all schools identified for 
improvement, and just 13 said these funds were sufficient to monitor the quality and 
effectiveness of supplemental educational service providers. About 80% of the 
districts surveyed also said they had costs associated with implementing NCLB that 
were not covered by federal funds, such as the costs of training teachers to meet 
NCLB qualifications, providing remedial services to students performing below grade 
level, and carrying out mandatory data collection and analysis. Although NCLB 
brought extra expenses and mounting federal demands, a large proportion of 
districts were receiving fewer Title I funds for school year 2004-05 than for the 
previous year. 

Where to Obtain 

www.cep-dc.orq 
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Center on Education Policy, 2006 

A shell game: Federal funds to improve schools 



Focus 

Analyzes how the process for reserving Title I funds specifically for improvement activities in 
identified schools affects the funding available to districts and states for all other Title I 
activities. 

Methodology 

Analyzed data from the U.S. Department of Education on school district Title I, Part A 
allocations for school year 2005-06. 

Major Findings 

• I nadequate funds for improvement. I n many states, funds available under the 
4% state set-aside for school improvement were insufficient to do the job because of 
inadequate NCLB appropriations and a "hold harmless" provision in the law that 
prevents school districts from losing Title I funds as a result of this set-aside. 

• Shell game. Complying with the school improvement set-aside became a "shell 
game" in 2005-06. To meet the set-aside without violating the hold harmless, states 
had to take money away from poorer school districts that were slated to receive 
increased Title I allocations and give the set-aside funds to other districts that may 
or may not have had as much poverty. The very districts that were supposed to get 
more Title I money because of their greater poverty actually received smaller or no 
increases due to the school improvement set-aside. 

• Shortfalls in improvement set-aside. In 2005-06, at least 10 states could not 
reserve the full 4% for school improvement required by law. In another 5 states, the 
4% set-aside used up almost all of the funding increases that school districts in the 
state had expected to receive. 

• Possible solution. This problem was expected to worsen in 2006-07 because of a 
decrease in Title I appropriations. But the problem could be solved if Congress would 
appropriate funds for school improvement through a special authorization in the Title 
I law for this purpose and would provide adequate total appropriations for NCLB. 

Where to Obtain 

www.cep-dc.org 
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Center on Education Policy, 2006 

Title I funds— Who's gaining, who's losing: School year 2006-07 update 



Focus 

Analyzes changes in allocations to states and school districts for school year 2006-07 under 

Title I, Part A, of ESEA, as amended by NCLB. 

Methodology 

Analyzed data on the final 2006-07 Title I, Part A allocations released by the U.S. 

Department of Education. 

Major Findings 

• Funding cuts and freezes. About 90% of the nation's school districts participating 
in Title I had their Title I funding cut or frozen for school year 2006-07. Half the 
states also lost funding. After the mandatory state set-aside for school improvement 
was applied, only 10% of districts were expected to receive Title I funding increases. 
The cause of these cuts and freezes was a stagnant level of federal appropriations for 
two years. 

• More poverty and more local responsibilities. These reductions and freezes for 
districts occurred at a time when the number of low-income children in the U.S. had 
risen and when the federal government was demanding more of schools and districts 
through NCLB. 

• School improvement funds. Because of stagnant appropriations, states were 
slated to receive only 60% of the funds they could have received under NCLB to 
assist schools identified for improvement. However, NCLB requires states to assist all 
such schools, regardless of how much funding is available. 

Where to Obtain 

www.cep-dc.org 
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Center on Education Policy, 2005 

Title I funds— Who's gaining and who's losing: School year 2005-06 update 



Focus 

Analyzes changes in allocations to states and school districts for school year 2005-06 under 

Title I, Part A, of ESEA, as amended by NCLB. 

Methodology 

Analyzed data on the 2005-06 Title I, Part A allocations released by the U.S. Department of 

Education 

Major Findings 

• Gains and losses. Although federal appropriations for Title I increased by 3% over 
the preceding year, about two-thirds of the school districts that participated in the 
Title I program received fewer funds for 2005-06 than they had the previous year. 
The number of districts losing Title I funding grew significantly between 2004 and 
2005. Nine states also had reductions in federal Title I funds, while 41 states, the 
District of Columbia, and Puerto Rico had gains. 

• Shifts in funding. The shifts in funding levels among school districts and states 
were mostly due to differences among districts in their numbers of poor children, as 
determined by annual estimates from the Census Bureau. These shifts were also 
related to the allocation formulas used to distribute new funds to districts with the 
highest concentrations of low- income children. As a result, more federal dollars were 
directed to districts with large concentrations of students from low- income families. 

• Increased poverty and more responsibilities. These reductions occurred at a 
time when the federal government was demanding more of districts and states 
through NCLB. In addition, the number of children in poverty grew by 6%, which 
meant that fewer Title I dollars were available for each poor child. 

Where to Obtain 

www.cep-dc.orq 
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The Civil Rights Project at Harvard University, 2006 

Massive responsibilities and limited resources: The state response to NCLB 



Focus 

Examines how states are meeting NCLB requirements and if they reallocate resources in 
ways that will meet the law's ambitious educational goals. Tests the assumption that state 
capacity is adequate and that reallocation of existing resources would suffice to meet the 
NCLB requirements. 

Methodology 

Analyzed states' responsibilities under NCLB and evidence about the strains on state 
capacity in many states. Examined how states negotiated tensions between limited 
resources and increased demands. 

Major Findings 

• Enormous responsibilities, few resources. A striking lack of resources was 
available for states to carry out the enormous responsibilities placed on them by 
NCLB and to accomplish the law's extraordinary goals. 

• States' good faith efforts. Facing immense challenges, state administrators did 
their best to comply with the law. States focused most on implementing the 
mandates for data collection, testing requirements, and other procedural aspects of 
the law because these were spheres of action that administrators could actually 
control. For the most ambitious goals of implementing large-scale interventions that 
produced fast and consistent gains, the law provided few resources. 

• Declining staff in some states. It was not unusual for state education agencies to 
experience a decline in the number of staff that started before NCLB and continued 
through the first years of implementation. 

• Division of authority among state players. The division of authority over 
education between governors, chief state school officers, and state legislatures 
added complexity to the administrative tasks of state education agencies. 

• Federal sanctions. States tried to avoid or limit the consequences of federal 
sanctions for non-compliance. The initiative to implement drastic school-level 
interventions increasingly passed from professionals to politicians as schools, 
teachers, and communities risked a loss of reputation and as educators were 
threatened with sanctions that often seemed disproportionate and counterproductive. 

Where to Obtain 

http://www.law.berkelev.edu/files/Massive_Responsibilities_Sunderman_Orfield.pdf 
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Education Sector, 2007 

Stiff armed: No Child Left Behind's unused funding flexibility 



Focus 

Examines the extent to which states and school districts are taking advantage of the 
provisions in NCLB that allow them the flexibility to take funds allocated for certain federal 
education programs and use them instead for other federal education programs. 

These flexibility programs include 1) the Rural Education Achievement Program (REAP Flex) 
for small rural districts; 2) the Local Flex program, which allows a competitively selected 
group of school districts to consolidate funding from certain programs; 3) the State Flex 
program, which allows a limited number of states, with permission of the Secretary of 
Education, to consolidate certain federal education funds; and 4) the Transferability 
provisions, which allow districts to transfer up to 50% of their initial federal formula 
allocations into and out of several specific programs. 

Methodology 

Analyzed data from on a U.S. Department of Education survey of school districts conducted 
in 2005-06. 

Major Findings 

• Limited use of flexibility provisions. The caps on the number of states and 
districts permitted to take part in the flexibility programs were unnecessary. Only 
one district participated in the Local Flex program, and no state participated in State 
Flex. Even with the broad qualification requirements for the Transferability option, 
only 16% of eligible districts participated. REAP-Flex was the only program with 
substantial participation; 51% of eligible rural districts participated. 

• Reasons for not using Transferability. Thirty-five percent of districts that didn't 
use the Transferability option said the amount of money eligible to transfer was 
insufficient to make an impact, while 44% said they already had enough funding 
flexibility prior to NCLB. Nearly half the non-participating districts said they lacked 
information or were confused about how the flexibility programs work. 

• Programs receiving funds. When money was reallocated it was overwhelmingly 
taken from the federal programs for teacher quality and safe and drug-free schools. 
It was redirected into programs for poor students, technology, and— most 
frequently— innovative programs. Perhaps not coincidentally, the innovative 
programs part of the law gives districts the widest latitude in use of funds. 

• Reasons for low participation. Several factors may contribute to the low 
participation in the flexibility programs. Federal funding often constitutes only a 
small percentage of local budgets, especially in more affluent communities. In 
addition, to protect funding for low-income children, flexibility provisions wall-off 
programs with the most funding, especially the Title I program. Other large 
categorical programs, such as vocational education and special education, are not 
included in the flexibility programs. Finally, many states and districts appear more 
interested in modifications to NCLB's accountability and teacher quality provisions 
than to the federal funding streams. None of the NCLB flexibility programs permit 
states or school systems to sidestep the law's accountability requirements. 
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• Future outlook. States need to do a better job informing schools about flexibility 
options and their benefits. Many districts, however, expect never to use the 
programs because they lack the need or do not see flexibility as a means to reaching 
accountability standards. 

Where to Obtain 

http://www.educationsector.orq/analvsis/analvsis show.htm?doc id=509534 
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U. S. Department of Education, Policy and Program Studies Service, 2007 

Evaluation of flexibility under No Child Left Behind: Vol. I— Executive summary of 
transferability, REAP flex, and local flex 



Focus 

Looks at the implementation of three programs under NCLB that afford school districts 
flexibility in their uses of federal education funds: 

1. The Rural Education Achievement Program (REAP Flex), which allows eligible small 
rural school districts to use funding from certain federal programs for other federal 
programs 

2. The Transferability provisions, which allow districts to transfer up to 50% of their 
initial federal formula allocations into and out of several specific programs authorized 
by ESEA 

3. The Local Flex program that allows a competitively selected group of school districts 
to consolidate funding from a set of eligible federal programs 

Methodology 

Conducted a survey in a nationally representative sample of 372 school districts, 
supplemented by 12 case study interviews in a small subset of districts. The data were 
collected during school year 2005-06. 

Key Findings 

• REAP Flex. REAP Flex was widely used. Rural districts were highly likely to 
participate in REAP Flex but less likely to participate in Transferability or Local Flex. 

• Receiving programs. Transferability and REAP Flex most commonly used flexibility 
to provide funds for programs under Title I -A (education of disadvantaged children) 
and Title V-A (innovative programs) of ESEA, as amended by NCLB. They most often 
took funds from Title I l-A (teacher quality) or Title I V-A (safe and drug-free schools). 

• Technical assistance source. Districts most commonly relied on their states for 
information about and technical assistance with Transferability and REAP Flex. 

• Reasons for participating. Districts usually participated in the three flexibility 
programs to target funds on areas of need that would allow them to make adequate 
yearly progress. Rural districts found the flexibility programs particularly useful 
because they often receive small allocations for programs and face funding 
constraints associated with declining student enrollments. 

• Reasons for not participating. Districts that did not participate in the flexibility 
programs gave as their two main reasons a lack of information or an inability to see 
clear benefits from the programs. The application process in Local Flex also 
discouraged districts from participating. Ensuring that districts have more accurate 
information and can better distinguish among the flexibility programs would likely 
increase participation. 

Where to Obtain 

http://www.ed.aov/ rschstat/eval/disadv/ flexibility/ execsum.htm I 
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U.S. Department of Education, Policy and Program Studies Service, 2007 

National Assessment of Title I: Final report, vol. I 



Focus 

Comprehensive study that describes the progress of states, districts, and schools through 
school year 2004-05 in implementing key provisions of Title I. Summarized below are the 
study's findings about funding. 

Methodology 

Drew on data from a set of implementation studies by the U.S. Department of Education. 
Data for these studies came from surveys conducted in a nationally representative sample 
of school districts, other state and local surveys, state performance reports, and the 
National Assessment of Educational Progress. 

Major Findings 

• What Title I funds buy. In 2004-05, most Title I funds were used for delivering 
instruction (73%), supporting the hiring of teachers (49%) and instructional aides 
(11%), and providing instructional materials and computers (12%). Sixteen percent 
of the funds were used for instructional support, and 11% for program 
administration and other support costs such as facilities and transportation costs. 

• Funds distribution. Title I funds were much more targeted to the highest- poverty 
districts than were state and local funds or federal education funds overall. Districts 
in the highest- poverty quartile received 52% of all Title I funds in 2004-05, more 
than double their share of state and local funds (22%) and greater than their share 
of federal education funds overall (39%). The highest- poverty schools (those with 
75-100% low- income students) received 38% of Title I school allocations, while 6% 
of the funds went to low- poverty schools. 

• Targeting to high-poverty districts. The share of funds received by the highest- 
poverty quartile of districts changed very little after enactment of NCLB (52% in 
2004-05, versus 50% in 1997-98). In the highest- poverty schools, Title I funding per 
low-income student remained unchanged between 1997-98 and 2004-05 after 
adjusting for inflation. The highest- poverty schools also continued to receive smaller 
Title I per pupil allocations ($558) than did low-poverty schools ($763). 

• Secondary school funding. Secondary schools were less likely to receive Title I 
funds than elementary schools; 40% of middle schools and 27% of high schools 
received Title I funds, compared with 71% of elementary schools. The average 
allocation per low-income student was also less in secondary schools— $502 in 
middle schools and $451 in high schools, compared with $664 in elementary schools. 

Where to Obtain 

http://ies.ed.qov/ncee/pdf/20084014_rev.pdf 
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Vermont Society for the Study of Education, 2007 

After five years: Revisiting the cost of the No Child Left Behind Act 



Focus 

Seeks to determine the degree to which the assertion is true that NCLB Act's 588 mandates 
are underfunded. 

Methodology 

Reviewed 17 state and local studies of new administrative costs— the costs to develop and 
administer new programs that didn't exist before NCLB. Also reviewed 55 state- level 
"adequacy" studies that sought to determine how much money would be needed to ensure 
children reach the standards states have developed for NCLB accountability. 

Major Findings 

• New state administrative costs. A sum exceeding all new monies from the federal 
government would be needed to fund state NCLB-mandated bureaucracies. Although 
new administrative costs of individual states varied, they represented approximately 
2.5% of all expenditures. Averaged across the nation, an estimated $24.4 billion 
would have been needed for fiscal year 2005 to cover these new administrative 
costs— more than the total Title I appropriation for that year. This would leave no 
new money for actually teaching children. 

• New local administrative costs. New local administrative costs were 
conservatively estimated to be another 2.5% of total spending. As a result, new 
money does not find its way into services for children. Whether the new and costly 
bureaucratic requirements of planning, communications, data analysis, and the like 
will have a direct and positive causal effect on improved test scores is a question yet 
to be answered. 

• Costs of adequately educating students. The NCLB adequacy studies indicate 
that education funding increases of 20% to 40% would be needed to adequately 
prepare children to meet standards. The median spending increase suggested by the 
adequacy studies is 27.5%. In estimating the costs of NCLB, the current adequacy 
studies also conclude, almost universally, that anti-poverty programs are essential if 
the nation is to meaningfully achieve the NCLB promise. 

• Total costs to meet NCLB requirements. Taken together, the 72 studies suggest 
that a national funding increase of more than 32%— or about $158.5 billion in fiscal 
year 2005 dollars— would be needed to meet the additional costs of fulfilling the 
requirements of NCLB. By any measure, the federal appropriations have fallen far 
short of the needs. Without such investments, children in low-spending, low- 
performing states and poor and non-English speaking children will be left behind. 

Where to Obtain 

http://www.vsse.net/?q=node/269 
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Arizona State University, Education Policy Studies Laboratory, 2005 

Evolution of federal policy and implications for No Child Left Behind for language minority 

students 



Focus 

Analyzes the implications of NCLB for limited English proficient (LEP) students. 

Methodology 

Reviewed legislative requirements of Titles I and III of NCLB. Reviewed history of major 

federal education programs for LEP students. 

Major Findings 

• Several negatives, one positive. NCLB marks a dramatic shift in guiding federal 
principles toward LEP students and could become "a recipe for leaving students 
behind," in the author's words. The sole positive outcome is that schools which have 
neglected LEP students can no longer afford to do because of requirements to 
disaggregate data by subgroups. 

• Testing policies. As of 2005, the majority of LEP students had to take state exams 
in a language in which they were not yet proficient, a requirement that encouraged 
English-only instruction. Although NCLB allows for exceptions and accommodations 
to the testing policies, the exceptions were limited as of 2005, and acceptable 
accommodations were not defined. 

• Excluding LEP students from accountability. While LEP students must be tested, 
states are using "psychometric gimmicks" to exclude their scores and help many 
schools avoid accountability for the LEP subgroup. This may create an illusion of 
success while the real needs of LEP students are being ignored. 

• Instructional goals. The goals for LEP programs are simply to mainstream students 
as soon as possible and teach them the content of the state tests. Schools are under 
pressure to raise test scores, so instruction has narrowly focused on the test and 
discouraged a focus on the true needs of LEP students. 

• English language instructional programs. The word "bilingual" was removed 
entirely from NCLB. Title III no longer makes a distinction between bilingual 
programs or special alternative instructional programs but only requires LEP students 
to be placed in "language instruction education programs." The use of teaching the 
student's native language is "optional." Bilingual education programs still allowed 
but only if state education leaders deem them as "scientifically based" and are willing 
to fund them. In some states, anti-bilingual education measures make it extremely 
difficult for schools to offer quality bilingual education programs. 

Where to Obtain 

http://epsl.asu.edu/epru/documents/EPSL-0501-101-LPRU.pdf 



CEP, Compendium of Major NCLB Studies Miscellaneous Studies 



Page 2 



Asian American Legal Defense and Education Fund, 2008 

Left in the margins: Asian American students and the No Child Left Behind Act 

Focus 

Examines how the education of Asian American students has fared under NCLB and 

discusses policy considerations that should be taken into account in reauthorizing NCLB. 

Methodology 

Reviewed existing data for specific Asian ethnic groups on demographic characteristics of 

Asian American students, enrollments of immigrant and ELL children, availability of ESL and 

bilingual teachers, dropout and college attendance rates, and parental education. 

Major Findings 

• Achievement struggles for many Asian American students. Contrary to 
stereotypes that cast Asian Americans as model high achievers, data disaggregated 
by Asian ethnic group and income level reveal that many Asian American students 
are struggling, failing, and dropping out of schools that ignore their needs. 

• Inadequate services for Asian American ELLs. Nearly one out of four (24%) 
Asian American students is an English language learner, and 12% of all ELLs are 
Asian Americans. Most school districts do not provide sufficient services for ELLs, 
especially those who speak a language other than Spanish. Asian language 
interpretation and translation services, bilingual programs, or translated assessments 
are hardly ever available even though they are essential. 

• NCLB policy recommendations. To better serve Asian Americans, NCLB should 
deemphasize high-stakes testing, encourage states to develop more native-language 
assessments, use multiple forms of assessment, create more bilingual education 
programs, address high dropout or push-out rates, increase professional 
development and hiring to ensure ELLs are instructed by appropriately trained 
teachers, and improve parental involvement and communication with immigrant 
parents, particularly those with little formal education. 

Where to Obtain 

http://www.aaldef.orq/docs/AALDEF LeftintheMarqins NCLB.pdf 
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Center on Education Policy, 2008 

Some perspectives from rural school districts on NCLB 



Focus 

Examines the impact of NCLB on student achievement and teacher quality in some rural 
districts and the challenges those rural districts face in complying with the Act. Analyzes 
differences in responses to NCLB between rural districts and urban or suburban districts. 

Methodology 

Based on a nationally representative survey of NCLB implementation in 349 responding 
districts and interviews with administrators in eight rural districts in various parts of the 
country. 

Major Findings 

• Causes of improved student achievement. The rural districts surveyed, like the 
urban and suburban districts, rated their own district policies and programs as more 
important causes of improved student achievement than the provisions of NCLB 
were. Exceptions were Reading First programs and NCLB school improvement plans, 
both which were rated as important or very important contributors to higher 
achievement by at least 72% of the rural districts that participated in these aspects. 

• Rural responses to NCLB. As a response to NCLB, rural districts participating in 
CEP's case studies had aligned their curriculum better with test content and had 
sharpened their focus on individualized instruction. 

• Achievement gaps. Sizeable percentages of the rural districts surveyed had 
achievement gaps for students with disabilities and low-income students. About 68% 
of districts reported having an achievement gap for students with disabilities in 
English language arts and math, and 50% reported having an achievement gap for 
low- income students. 

• Subgroups too small to track. A smaller share of rural districts than of urban or 
suburban districts reported having achievement gaps for minority students or English 
language learners, but this was because they enrolled too few of these students to 
calculate gaps under NCLB. At least two-thirds of the rural districts surveyed— or 
more, depending on the subgroup— reported having too few racial, ethnic, or 
language minority students to track separately for purposes of determining AYP. 

• Teacher requirements. NCLB's highly qualified teacher requirements appeared to 
have had a limited impact on teacher recruitment and retention in most rural 
districts. About 47% of rural districts reported no outcomes of these requirements. 
Rural districts reported having the most difficulty complying fully with the highly 
qualified requirements for secondary school science and math teachers. 

Where to Obtain 



www.cep-dc.orq 
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Center on Education Policy, 2007 

Reading First: Locally appreciated, nationally troubled 



Focus 

Examines the perceived effectiveness of Reading First, the challenges of implementing the 
program, its coordination with Title I, the expansion of the program, and the accuracy and 
usefulness of program evaluations. 

Methodology 

Based on 2006-07 data from an annual survey of 50 states, a nationally representative 
survey of 349 responding school districts, and case study interviews with district- and 
school-level administrators in nine school districts. 

Major Findings 

• Local value. Despite misconduct and mismanagement at the national level and in 
some states, the Reading First program has had a meaningful impact at the local 
level. Schools and district appear to have implemented Reading First as intended, 
and some evidence suggests the program may have a positive impact on 
achievement. 

• Views about effectiveness. The majority of states (82%) reported that Reading 
First professional development is very or moderately effective in raising student 
achievement, while 78% said that Reading First curriculum and assessment 
materials are very or moderately effective. Among school districts with Reading First 
grants, 69% reported that Reading First's assessment systems were important or 
very important causes of increased student achievement, and 68% said Reading 
First's instructional programs were important or very important causes. 

• Changes in reading instruction. Sixty-seven percent of districts participating in 
the program reported making changes in how teachers teach reading. These changes 
included purchasing new materials. 

• Effect beyond participating districts and schools. The impact of Reading First 
goes beyond the 13% of districts and 6% of schools nationwide that participated in 
the program in 2006-07. More than half of Reading First districts reported using 
elements of Reading First in schools that did not receive Reading First funds and in 
the upper grades (Reading First targets early elementary students). An exception 
was the element of a reading coach, which fewer districts used in non-Reading-First 
schools or grades, perhaps because the cost was prohibitive. Similarly, states 
reported that more than 3,000 non-Reading-First districts participated in state- led 
Reading First professional development. 

• Coordination with Title I . Most states (80%) and most districts (75%) with 
Reading First grants indicated that Reading First is well coordinated with Title I. 

Where to Obtain 

wwwf. cep-dc.orq 
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Center on Education Policy, 2006 

Keeping watch on Reading First 



Focus 

Discusses the implementation and impact of the federal Reading First program as of 2006. 

Methodology 

In 2006, conducted a survey of all 50 states and a survey of a nationally representative 
sample of 417 school districts that were participating in Title I. Did case study interviews 
with staff in 38 districts and 42 schools. Reviewed state and national testing data and other 
evaluations of Reading First. 

Major Findings 

• Significant impact. Reading First has had a significant impact on reading 
curriculum, instruction, assessment, and scheduling in participating schools. The 
program has also affected non- participating schools and districts. Many districts have 
expanded Reading First instructional programs and assessments into other schools, 
and many states have used Reading First state funds for professional development 
and technical assistance that benefits other districts. 

• Cause of achievement gains. Of the 35 states that reported student achievement 
in reading was improving, 19 states cited Reading First instructional programs as an 
important or very important cause of these gains, and 16 said that Reading First 
assessments were an important or very important cause. In addition, 97% of 
districts that received Reading First subgrants and also had achievement gains cited 
Reading First's instructional program as an important or very important cause of this 
improvement. 

• Change in reading programs. Sixty percent of Reading First districts reported they 
had to change their reading program in order to qualify for a Reading First subgrant. 
In addition, 86% of districts with Reading First subgrants required their elementary 
schools to devote a specified amount of time to reading, significantly more districts 
than the 57% of non- Reading First districts that had such a requirement. The 
average amount of time the two types of districts devoted to reading, however, was 
similar— about an hour and a half. 

• Title I coordination. Seventy-six percent of responding states and 80% of districts 
with Reading First grants reported that they coordinated Reading First and Title I 

• Lack of coordination with Early Reading First. The majority of states (65%) 
reported that Reading First was not coordinated with Early Reading First, a federal 
program at improving pre-reading and language skills in pre- kindergarten children. 

Where to Obtain 



www.cep-dc.orq 
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Center on Education Policy, 2005 

Ensuring academic rigor or inducing rigor mortis? Issues to watch in Reading First 

Focus 

Identified issues in the Reading First program in school year 2004-05 that warranted special 
attention. 

Methodology 

Collected data through a 2004-05 survey of all 49 states, a nationally representative survey 
of 314 school districts, case studies of 36 geographically diverse districts and 37 schools, 
and three national forums. Also conducted an overview of all state Reading First grant 
applications, did an in-depth review of 15 randomly selected state applications, and 
reviewed revisions to state grant applications from 10 representative states in the South, 
East, and West. 

Major Findings 

• ED enforcement. In school year 2004-05, 40 out of 49 responding states said that 
the U. S. Department of Education was enforcing Reading First strictly or very 
strictly. 

• Impact of Reading First. Flalf of the districts that received Reading First grants 
reported changing their reading program to qualify for grants. Educators in case 
study schools also reported that Reading First was affecting reading instruction, 
although views varied as to how helpful Reading First was in improving reading 
instruction. 

• Reading First assessments. States were remarkably consistent in their selection of 
specific instruments for assessing students' reading progress. Furthermore, CEP's 
review of a 10 initial and final state grant applications showed that several states 
added two specific instruments to their applications after the peer review process. It 
was not clear whether this was due to pressure from reviewers or states learning 
more about the quality of the instruments. 

• Coordination with other reading programs. Coordination with other reading 
initiatives was not a priority in 2004-05, although CEP's review of final grant 
applications from 15 randomly selected states showed that all of these states had at 
least 4 other reading initiatives in place and one state had as many as 16. 

Where to Obtain 

www.cep-dc.orq 
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Government Accountability Office, 2007 

Reading First: States report improvements in reading instruction, but additional procedures 
would clarify Education's role in ensuring proper implementation by states 



Focus 

Examines the implementation of the Reading First program, including changes in reading 
instruction, criteria states used to award subgrants to districts, difficulties states faced 
during implementation, and guidance, assistance, and oversight provided to states by the 
U.S. Department of Education. 

Methodology 

Drew from a Web-based survey of Reading First directors in 50 states and D.C.; site visits; 
and interviews with federal, state, and local education officials and providers of reading 
programs and assessments. Also reviewed ED data for each state on Reading First districts' 
eligibility, applications, and funding awards. 

Major Findings 

• Changes due to Reading First. States reported making a number of changes to, 
as well as improvements in, reading instruction since the implementation of Reading 
First. These included an increased emphasis on the five key components of reading 
(phonemic awareness, phonics, vocabulary, fluency, and comprehension), 
assessments, and professional development. States also reported more classroom 
time being devoted to reading activities. Flowever, the publishers interviewed noted 
only limited changes to instructional material. Similarly, states reported that few 
changes occurred in their approved reading lists. 

• Award of subgrants. States awarded Reading First subgrants using various 
eligibility and award criteria, and some states reported difficulties with implementing 
key aspects of the program. According to ED, over 3,400 districts were eligible to 
apply for subgrants in the states' first school year of funding. Of these districts, 
nearly 2,100 applied for and nearly 1,200 districts received Reading First funding. 

• ED influence. ED officials made various resources available to states during the 
application and implementation processes, and states were generally satisfied with 
the guidance and assistance they received. Flowever, ED developed no written 
policies and procedures to guide Departmental officials and contractors in their 
interactions with state officials and guard against officials mandating or directing 
states' decisions about reading programs or assessments. Some state officials 
surveyed reported receiving suggestions from ED officials or contractors to adopt or 
eliminate certain reading programs or assessments. Similarly, the Inspector General 
reported in September 2006 that the Department intervened to influence a state's 
and several school districts' selection of reading programs. 

• ED monitoring. While ED officials laid out an ambitious plan for annual monitoring 
of every state's implementation, they did not develop written procedures guiding 
monitoring visits. As a result, states did not always understand monitoring 
procedures, timelines, and expectations for taking corrective actions. 

Where to Obtain 

http://www.aao.aov/new.items/d07161.pdf 
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Northwest Regional Educational Laboratory, 2008 

Does Reading First work? Data trends in five Western states 



Focus 

Analyzes the impact of Reading First on student achievement and other areas in five 
western states. Sought to provide a more nuanced picture of the impact of the program 
than the federally funded Reading First impact study had done. 

Methodology 

Summarized data from annual statewide evaluations of Reading First implementation and 
outcomes conducted in 2003 and 2004 by the Northwest Regional Educational Laboratory 
(NWREL) in Alaska, Montana, Washington, and Wyoming, and by Arizona State University 
and NWREL in Arizona. Also analyzed student achievement data from the Dynamic Indicator 
of Early Literacy Skills (DIBELS) test from 2003-04 to 2007. 

Major Findings 

• Achievement gains. The percentage of students performing "at benchmark" on the 
DIBELS assessment improved steadily. The percentage of students performing at the 
lowest ("intensive") level on the same assessment decreased over time. 

• Influence in non-Reading First schools. Across the five states, non- Reading First 
schools in districts with Reading First grants frequently implemented many Reading 
First program components. 

Where to Obtain 

http://www.nwrel.orq/crea/pdf/rf-trends.pdf 
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U.S. Department of Education, Policy and Program Studies Service, 2008 

Reading First impact study: Final report 



Focus 

Examines the impact of Reading First funding on students' reading achievement and 
classroom instruction, and the relationship between the degree of implementation of 
scientifically based reading instruction and students' reading achievement. 

Methodology 

Studied a sample of 17 school districts in 12 states and one statewide program (18 sites), 
as well as 238 schools within these districts (including some schools that received Reading 
First grants and some that did not). Analyzed scores from the reading comprehension 
subtest of the SAT-10 for students in grades 1 through 3 over three years (2004-05 through 
2006-07). Also analyzed results from an assessment of reading achievement in decoding 
given to 1^‘ graders in spring 2007. Conducted formal classroom observations of reading 
instruction over three years (2004-05 through 2006-07). Administered a survey of 
education personnel in spring 2007 about program implementation. 

Major Findings 

• No significant achievement impact except in grade 1 decoding. Reading First 
did not produce a statistically significant impact on student reading comprehension 
test scores in grades 1, 2, or 3. Reading First did produce a positive, statistically 
significant impact on decoding skills among 1^‘ grade students tested in spring 2007. 

• More time on essential components. In grades 1 and 2, Reading First produced a 
positive and statistically significant impact on the amount of instructional time spent 
on the five essential components of reading instruction promoted by the program 
(phonemic awareness, phonics, vocabulary, fluency, and comprehension). 

• Impact on reading instruction. Reading First produced positive, statistically 
significant impacts on multiple practices promoted by the program, including 
professional development in scientifically based reading instruction, support from 
full-time reading coaches, amount of reading instruction, and availability of supports 
for struggling readers. 

• Exploratory analyses. The study also explored factors that might account for the 
impacts observed; these findings are suggestive and were not rigorously tested: 

> There was no consistent pattern of effects over time in reading comprehension at 
any of the grades studied or in the impact of the program on reading instruction 
in grade 1. There appeared to be a systematic decline over time in the impact of 
the program on reading instruction in grade 2. 

> There was no relationship between reading comprehension and the number of 
years a student was exposed to Reading First. 

> There were no statistically significant variations among program sites in the 
impact of Reading First on reading comprehension or instruction. 

> There was a positive association between time spent on the five essential 
components of reading instruction and SAT- 10 reading comprehension scores. 

Where to Obtain 

http://ies.ed.qov/ncee/pubs/20094038/index.asp 
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U.S. Department of Education, Policy and Program Studies Service, 2008 

Reading First implementation evaluation: Final report* 



Focus 

Evaluates the implementation and impact of the federal Reading First program. Compares 
instruction and achievement in schools that receive Reading First subgrants and schools that 
did not. Includes an additional year of follow-up data beyond that available in the interim 
report of this study. 

Methodology 

Conducted surveys of K-3 teachers, principals, and reading coaches in spring 2005 and 
2007. Surveys were administered to a nationally representative sample of Reading First 
schools and to non- Reading First schools with Title I schoolwide projects. Also analyzed 3'^'^ 
and grade reading scores on 2005 state assessments and composite measures relating 
to schools' implementation of the program based on 2005 survey questions. 

Major Findings 

• Differences in Reading First and non-Reading First schools. Reading First 
schools and non- Reading First schools differed in several ways. Reading First schools 
devoted more time to reading instruction in K-3 classrooms than non-Reading First 
schools. Reading First schools were also more likely to a) have reading coaches who 
assisted teachers in implementing their reading programs; b) use reading materials 
aligned with scientifically based reading research; c) use assessments to guide 
instruction; d) place struggling readers into intervention services; and e) have their 
teachers participate in reading- related professional development. 

• Reading First activities. Non- Reading First schools increasingly reported 
conducting activities aligned with the principles of Reading First, including a) the 
provision of assistance to struggling readers; b) teacher knowledge and use of 
materials and strategies aligned with scientifically based reading research; and c) 
staff participation in professional development in the five dimensions of reading 
instruction (phonemic awareness, phonics, vocabulary, fluency and comprehension). 

• I mpact on achievement. Analyses of states' reading assessment scores showed 
limited but statistically significant evidence that successive cohorts of 3'^'^ and 4*^^ 
grade students in Reading First schools improved their reading performance over 
time more quickly than did their counterparts in non-Reading First Title I schools. 

• Reading First-aligned activities and achievement. The study analyzed the 
relationship between schools' 3'”'^ grade reading scores and four composite measures 
intended to characterize schools' implementation of activities aligned with Reading 
First: classroom reading instruction, strategies to help struggling readers, 
participation in professional development, and uses of assessment to inform 
instruction. Only one of the four composite measures— teachers' use of strategies to 
help struggling readers— showed a positive and statistically significant relationship to 
3'^'^ grade reading achievement. 

Where to Obtain 

http://www.ed.gov/ rschstat/eval/other/readinqfirst-final/ readinqfirst-final.pdf 



*Also see the summary for ED's Reading First I mpiementation Evaiuation: Interim Report of 2006. 
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U.S. Department of Education, Policy and Program Studies Service, 2006 

Reading First implementation evaluation: Interim report* 



Focus 

Examines implementation of the Reading First program in districts and schools and 

difference in reading instruction between Reading First and non-Reading First Title I schools. 

Methodology 

Conducted surveys in spring 2005 of K-3 teachers, principals, and reading coaches in 

nationally representative samples of 1,092 RF schools and 541 non-RF Title I schools. 

Carried out interviews with RF state coordinators. Reviewed states' applications for Reading 

First awards, data from the awards database, and data from the U.S. Department of 

Education's School- Level State Assessment Score database. 

Major Findings 

• General implementation. Reading First schools appeared to be implementing the 
major elements of the program as intended by the legislation. Classroom reading 
instruction in Reading First schools was significantly more likely to adhere to Reading 
First law than instruction in Title I schools. 

• More intervention. Reading First teachers in kindergarten, 2 '^^, and 3'^'^ grades were 
significantly more likely than their counterparts in Title I schools to place their 
struggling students in intervention programs. 

• Funding sources. Reading First schools received both financial and nonfinancial 
support from a variety of external sources. During school year 2004-05, the median 
annual amount of funds Reading First schools received to implement their reading 
program was $138,000. Reading First schools have multiple external resources, in 
addition to Reading First funds, to implement reading programs. 

• Assessment selection. Assessment played an important role in reading programs 
in both Reading First and non-Reading First Title I schools. Reading First schools 
received more outside assistance in selecting assessments than Title I schools. There 
were some differences in the types of assessments that teachers in Reading First 
schools and teachers in non-Reading First Title I schools found useful. Teachers in 
Reading First schools were more likely to report applying assessment results for 
varied instructional purposes (such as grouping students, monitoring progress, and 
identifying struggling readers) than their Title I counterparts. 

• Reading coaches. Principals in Reading First schools were significantly more likely 
to report having a reading coach than were principals of non-Reading First Title I 
schools. 

• Professional development. Reading First staff received significantly more 
professional development than did Title I staff. 

Where to Obtain 

http://www.ed.aov/ rschstat/eval/other/readinqfirst- interim/ readinqfirst.pdf 



Also see the summary for ED's Reading First Implementation Evaluation: Final Report of 2008. 
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U.S. Department of Education, Office of I nspector General, 2007 

The Department's administration of selected aspects of the Reading First program: Final 

audit report 



Focus 

Determines whether the U. S. Department of Education carried out its role in accordance 
with applicable laws and regulations in administering Reading Leadership Academies (RLAs) 
and related meetings and conferences, awarding the National Center for Reading First 
Technical Assistance (NCRFTA) contract, and administering its Web site and guidance for the 
Reading First program. 

Methodology 

Interviewed officials in the ED Reading First Program Office and Office of General Counsel, 
the National Institute for Literacy, and RMC Research Corporation (an ED contractor). Also 
interviewed speakers and 17 randomly selected participants involved in the three RLAs. 
Reviewed RLA handbooks and video footage and the NCRFTA contract. 

Major Findings 

• Web site and guidance. ED generally administered its Reading First Web site and 
its April 2002guidance for the Reading First Program, in accordance with applicable 
laws and regulations. 

• Reading Leadership Academies. ED did not have controls in place to ensure that 
the RLAs complied with the Department of Education Organization Act and the 
curriculum provisions of NCLB. Specifically, the audit found that the "Theory to 
Practice" sessions at the RLAs focused on a select number of reading programs and 
that the RLA handbook and guidebook appeared to promote a specific assessment, 
the Dynamic Indicators of Basic Early Literacy Skills (DIBELS). 

• Technical assistance contract. With regard to RMC Research Corporation's (RMC) 
technical proposal for the NCRFTA contract, the audit concluded that ED did not 
adequately assess issues of bias and lack of objectivity when approving individuals to 
be technical assistance providers before and after the NCRFTA contract was awarded. 

• Recommendations. The Inspector General's Office recommended that ED establish 
controls to ensure it does not promote or appear to endorse specific curriculum in its 
conference materials and related publications and ensure that it adequately assesses 
possible bias and lack of objectivity in possible contractors. 

Where to Obtain 

http://www.ed.aov/about/offices/list/oia/auditreports/a03a0006.pdf 
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U.S. Department of Education, Office of I nspector General, 2006 

The Reading First program's grant application process: Final inspection report 



Focus 

Determines whether the U. S. Department of Education selected an expert panel to review 
Reading First grant applications in accordance with NCLB and adequately screened the panel 
members for possible conflicts of interest. Also examines whether the expert review panel 
adequately documented its reasons for stating that an application was not ready for funding 
and reviewed the applications in accordance with established criteria, consistently applied. 

Methodology 

Fieldwork began in September 2005, and concluded in J uly 2006. I nterviewed ED staff in 
the Reading First program office and the Office of General Counsel and reviewed ED 
guidance and correspondence related to the Reading First program. Also reviewed the 
Reading First applications from Connecticut, Georgia, Kentucky, Massachusetts, Michigan, 
Nevada, New York, North Carolina, North Dakota, Virginia, Wisconsin, and Puerto Rico. 

Major Findings 

• Conflicts of interest. The process developed by ED to screen expert review 
panelists for conflicts of interest was not effective. The resumes of six panelists 
revealed significant professional connections to a teaching methodology that requires 
the use of a specific reading program. ED did not identify any of these connections in 
its screening process and, therefore, would not have been in a position to deal with 
potential conflicts of interest if a state application included this specific program. 

• Peer review process. The expert review panel adequately documented its reasons 
for stating that an application was not ready for funding, but ED substituted a 
Department-created report for the panel's comments. As a result, ED did not follow 
its own guidance for the peer review process. Five states' applications were funded 
without documentation that they met all of the criteria for approval. ED included 
requirements in the criteria used by the expert review panels that were not 
specifically addressed in NCLB. 

• Application review. The expert review panel appeared to have reviewed 
applications in accordance with ED- developed criteria and to have applied the 
criteria consistently. Flowever, these criteria included language that was not based 
on the statutory language; as a result, state applications were forced to meet 
standards not required by the law. 

• Failures of accountability. ED program officials failed to maintain a control 
environment that exemplified management integrity and accountability. In 
implementing the Reading First Program, ED officials obscured the statutory 
requirements of ESEA; acted in contravention of the GAO Standards for Internal 
Control in the Federal Government; and took actions that called into question 
whether they violated the Department of Education Organization Act, which prohibits 
ED officials from exercising direction, supervision, or control over curricula or 
instructional programs. Specifically, the Office of Inspector General found that the 
Department: 

> Developed an application package that obscured the requirements of the statute; 

> Took action with respect to the expert review panel process that was contrary to 
the balanced panel composition envisioned by Congress; 
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> Intervened to release an assessment review document without the permission of 
the entity that contracted for its development; 

> Intervened to influence a state's selection of reading programs; and 

> Intervened to influence reading programs being used by school districts after the 
application process was completed. 

Where to Obtain 

http://www.ed.qov/about/offices/list/oiq/aireports/il3f0017.pdf 
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U.S. Department of Education, Policy and Program Studies Service, 2005 

Analysis of state K-3 reading standards and assessments 



Focus 

Evaluates the degree to which state reading content standards for K-3 students reflected 
expectations for learning in the five essential areas of effective reading instruction 
(phonemic awareness, phonics, vocabulary, fluency, and comprehension); and determines 
the extent to which state assessments administered in grades K-3 played a role in the 
measurement of Reading First outcomes in the five areas. 

Methodology 

Conducted an expert review in J anuary 2004 of state reading content standards for grades 
K-3 from a random sample of 20 states. Also conducted a systematic review of approved 
Reading First applications for all states and the District of Columbia to determine which K-3 
statewide assessments were being used as measures of the five essential areas of reading 
instruction. 

Major Findings 

• Representation of essential elements in standards. Reading comprehension is 
the most represented of the essential elements in state K-3 reading content 
standards with an average of 57 standards per state. This was followed by 
vocabulary (19 standards per state), phonics (16), fluency (6), and phonemic 
awareness (6). Most standards representing each element were judged to be placed 
at the appropriate grade in most of the states studied. A few states were found to 
have placed standards representing phonemic awareness and phonics at too high of 
a grade level. 

• Visibility of elements. States that had organized larger numbers of their K-3 
reading standards to make the five essential elements more visible were judged to 
represent these elements better. 

• Inadequate attention to expected outcomes. With the possible exception of 
vocabulary and comprehension in grade 3, statewide reading assessments in 2003- 
04 did not significantly address expected student outcomes from reading instruction 
in the five essential areas. 

• Relationship of standards and assessments. There is a slight relationship 
between how state standards represent the five essential elements of reading 
instruction and how state assessments represent them. States that identified their 
statewide reading assessments as Reading First outcome measures tended to have 
more reading standards that visibly represented the five elements. 

Where to Obtain 

http://www.ed.aov/ rschstat/eval/other/ readinq/state- k3-reading.pdf 
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Center on Education Policy, 2008 

A call to restructure restructuring: Lessons from the No Child Left Behind Act in five states* 

Focus 

Synthesizes findings from CEP's multiyear studies of NCLB restructuring in five states with 
relatively large numbers of schools in restructuring: California, Georgia, Maryland, Michigan, 
and Ohio. Also looks at the number of schools in restructuring across the whole nation and 
restructuring policies and practices at the state and local levels in these five states. 

Methodology 

Collected data through document reviews, interviews with state and local officials, and case 
studies across five states in 19 districts and 42 schools within those districts. Focuses on 
restructuring activities through school year 2007-08. 

Major Findings 

• Number of schools nationally in restructuring. More than 3,500 schools, or 
about 7% of all Title I schools, were in the planning or implementation phase of 
restructuring in school year 2007-08; this represents an increase of more than 50% 
over the preceding year. In the five states studied, the vast majority of restructuring 
schools were in urban districts. 

• Staying in restructuring. I n the five states studied, just 19% of the schools that 
were implementing restructuring had made AYP based on 2006-07 tests. Some had 
been in restructuring for as long as four years. 

• Most common restructuring option. The so-called "any-other" restructuring 
option— taking any major action to fundamentally change school governance, other 
than the four more specific options in NCLB law— was the most popular federal 
restructuring option. Between 86% and 97% of restructuring schools in the five 
states chose the "any other" option. However, interpretations and implementation of 
this option varied widely among the five states and 42 case study schools. 

• School improvement set-aside. The amount of funding available per school under 
the Title I 4% state set-aside for school improvement varied widely across the five 
states studied. These states also had very different methods of distributing the set- 
aside funds, although all five states sent a sizeable share to school districts. 

• State supports. The five states studied varied greatly in the supports they offered 
restructuring schools. Four states sponsored extra professional development to help 
schools and districts with restructuring and school improvement. Three of the states 
provided on-site technical assistance to some restructuring schools, with more 
intense support and monitoring to schools that had been in restructuring for multiple 
years. Two states offered extra professional development specifically for some 
principals of restructuring schools, and two provided on-site leadership coaches or 
facilitators for restructuring schools. 

• Results of restructuring. Outcomes varied by state not but strategy. Significantly 
larger percentages of restructuring schools in Michigan and Georgia made AYP than 



‘CEP's individual annual reports about each of these five states are available at http://www.cep-dc.org. 
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in the other states; these variations were most likely related to differences among 
states in their content standards, test difficulty, and definitions of "proficient" 
performance, as well as differences in funding and state policies for restructuring. 
None of the five federal restructuring options was associated with a greater likelihood 
of a school making AYP. I n addition, none of the staff interviewed in schools that had 
exited restructuring could point to a single strategy they believed was the key to 
improving student achievement. 

• Common strategies. Even schools that chose different federal restructuring option 
used some common strategies to raise student achievement. All 42 case study 
schools used data for instructional decision making. The majority also provided 
tutoring to struggling students and employed an instructional or leadership coach. 

• Staff replacement. Replacing all or most of the school staff sometimes had 
unintended negative consequences in restructuring case study schools. Some 
principals reported being unable to replace staff with qualified teachers, while others 
spent so much time hiring staff they had little time to plan for the school year. Union 
regulations sometimes compromised effective restaffing. Most schools that replaced 
staff successfully had a large pool of applicants, a plan that allowed the school to 
overcome its past reputation as a "failing" school, help from the teachers' union, and 
effective hiring systems that did not rely on principals alone. The experience of some 
Maryland schools suggests that problems with staff replacement decrease in time. 

• Actions to raise achievement. The case study schools that missed AYP targets 
solely due to the performance of student subgroup(s) typically provided special 
programs to help raise achievement for students in traditionally underserved 
subgroups. But their focus on subgroups was less intense than might be expected. 
Even the case study schools that missed targets solely due to subgroups still devoted 
considerable resources to initiatives to raise achievement for all students. 

• Maintaining achievement. Principals and teachers at case study schools that had 
exited restructuring expressed concern about maintaining achievement gains and 
continuing to make AYP as targets kept rising toward 100% proficiency. Maintaining 
student achievement was also seen as difficult because schools that exited 
restructuring often lost some of their resources, including special funding for school 
improvement. 

Where to Obtain 

http: //www.cep-dc.orq 
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Center on Education Policy, 2007 

Moving beyond identification: Assisting schools in improvement 



Focus 

Examines what kinds of assistance schools identified for NCLB improvement receive and 
how effective state and district officials believe that assistance to be. Focuses on activities 
during school years 2005-06 and 2006-07. One of a series of CEP reports on the fifth year 
of NCLB implementation. 

Methodology 

Collected data through an annual survey of 50 state departments of education; a nationally 
representative annual survey of 349 responding school districts; and case studies involving 
interviews with district and school officials in 12 school districts. 

Major Findings 

• District with schools in improvement. About 18% of school districts had schools 
in improvement during school year 2005-06. Significantly more urban districts (47%) 
had schools in improvement than suburban (22%) or rural districts (11%). 

• State view of most effective strategies. States rated the following four 
frequently used state strategies as most effective for improving schools: professional 
development through Reading First (81% of 48 responding states), Reading First 
curriculum and assessment materials (79% of 47 states), alignment of curriculum 
and instruction with standards and/or assessments (76% of 46 states), and special 
grants to districts to support school improvement (74% of 47 states). 

• District views. A high percentage of districts (69% in reading and 71% in math) 
rated their own policies for improving schools as "important" or "very important" 
factors in raising student achievement— much higher than the share that rated state 
policies as important or very important (45% in reading and 35% in math). A 
relatively high share (68% or more) of districts with Reading First grants rated 
Reading First assessment systems and instructional programs as important or very 
important factors in raising achievement. In math, no federal policy was rated as 
important or very important by a large proportion of districts. 

• Federally mandated or encouraged actions. Actions that NCLB requires or 
encourages districts to take with schools in improvement— extending the school day, 
implementing new curricula, engaging in school improvement planning, and 
appointing outside experts to advise the school— were typically viewed by districts as 
helpful, but many districts said they lacked funding to take some of these actions. 

• Additional strategies. Districts reported that the most effective strategies to assist 
schools in improvement included "increasing the use of student achievement data to 
inform instruction and other decisions" (97% used this strategy), "increasing the 
quality and/or quantity of teacher and principal professional development" (94%), 
and "improving the school planning process" (93%). 

• Gap-closing measures. Districts took a variety of actions to close achievement 
gaps. The most common strategies included providing tutoring for low-performing 
subgroups, improving collaboration between special education and regular classroom 
teachers, and training teachers in specific methods to address the academic needs of 
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low-performing subgroups. Both districts and states reported that gaps for students 
with disabilities were the most difficult to close. 

• Funding for school improvement. Due to NCLB's "hold harmless" provision, not all 
states were able to set aside the full 4% of Title I funds for school improvement as 
directed by the law. Many states reported using state funds to address some goals of 
NCLB, but noted that state funding was uncertain. The majority of districts studied 
reported being unable to do certain things to assist schools in improvement because 
of lack of funds. 

Where to Obtain 

http: //www.cep-dc.orq 
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Center on Education Policy, 2006 

From the capital to the classroom: Year 4 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2004-05 and 2005-06. Summarized below 
are the study's findings about school improvement strategies. 

Methodology 

Collected data through a survey of all 50 states, a nationally representative survey of 299 
school districts, case studies of 38 geographically diverse districts and 42 schools, three 
national forums, and six special analyses of critical issues in implementing NCLB. 

Major Findings 

• State improvement strategies. The strategies used in 2005-06 by the largest 
number of states to improve student achievement in schools identified for 
improvement were making "special grants to districts to support school improvement 
efforts" (45 states) and "aligning curriculum and instruction with standards and 
assessment" (44 states). These were the same strategies that states reported to be 
moderately or very successful in raising student achievement. 

• District improvement strategies. Among school districts, the most popular 
strategies used in 2005-06 to improve achievement in schools identified for 
improvement were using research to inform decisions about improvement strategies 
(used by 96% of districts), aligning curriculum and instruction with standards and 
assessments (96%), and increasing the use of student achievement data to inform 
instruction and other decisions (95%). These same strategies were reported to be 
moderately or very successful in raising student achievement by at least three- 
quarters of school districts. 

Where to Obtain 

http: //www.cep-dc.orq 
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Center on Education Policy, 2005 

From the capital to the classroom: Year 3 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2003-04 and 2004-05. Summarized below 
are the study's findings about school improvement strategies. 

Methodology 

Collected data through a survey of 49 states, a nationally representative survey of 314 
school districts, case studies of 36 geographically diverse districts and 37 schools, three 
national forums, and four special analyses of critical issues in implementing NCLB. 

Major Findings 

• District improvement strategies. To boost performance in schools identified for 
improvement., virtually all of the districts we surveyed in 2004-05 said they had 
increased their use of student test data to inform instruction (100% of districts), 
aligned curriculum and instruction with standards and assessments (99%), and 
provided extra or more intensive instruction to low-achieving students (99%). About 
96% of the districts also said they had increased the quality and quantity of teacher 
professional development. In addition, CEP's case studies suggested that many 
districts were using "coaches" to revise reading and math curriculum, help teachers 
introduce more effective teaching strategies, or change the culture of 
underperforming schools. 

• Corrective action and restructuring strategies. More dramatic changes were 
occurring in 2004-05 in schools identified for corrective action or restructuring. Some 
districts reported that they had replaced school principals and teachers. Others 
closed low-performing schools, in some cases reopening them as new schools with 
different staff and management structures (and a clean slate for demonstrating AYP). 

Where to Obtain 

http: //www.cep-dc.orq 
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Council of the Great City Schools, 2006 

No Child Left Behind in America's Great City Schools: Five years and counting 



Focus 

Comprehensive study that examines the status and implementation of NCLB in the large 
urban districts that comprise the Council of the Great City Schools. Summarized below are 
the study's findings about schools improvement, corrective action, and restructuring. 

Methodology 

Based on data from 36 large urban districts, enrolling more than 5.1 million students, that 
responded to a survey by the Council of Great City Schools of its 66 member districts. The 
survey focused on NCLB implementation from school years 2002-03 through 2005-06. 

Major Findings 

• Schools and districts in improvement. Altogether, 2,203 schools in the 36 
districts studied, or almost 30% of all schools in these districts, were in various 
stages of NCLB improvement in 2005-06. This marked an increase over the 975 
schools in improvement in 2002-03. About 26.1% of the nation's schools in 
improvement were located in one of these 36 cities. 

• District improvement. Twenty-four of the 36 districts studied were identified for 
improvement in 2005-06. Some of the districts had been required by their state to 
implement new curricula, undergo restructuring, or replace some personnel. Four 
districts said that their states had reduced or deferred some funding. Eighteen of the 
districts received technical assistance from their states; most of the districts rated 
this assistance as "moderate" in quality. Districts in improvement pursued several 
systemic steps to raise achievement, including setting clearer goals, improving 
planning, adopting more stringent personnel accountability and evaluation systems, 
upgrading curriculum, instituting better instructional interventions, more closely 
aligning curriculum and supplemental materials with state standards and 
assessments, providing more targeted professional development, using more 
benchmark testing, focusing more on data analysis, using more intensive coaching 
and instructional monitoring, and similar strategies. 

• Corrective action. Approximately 458 schools in the 36 districts responding to the 
Council's survey were in the corrective action stage of NCLB. The majority of the 
districts provided technical assistance to schools in corrective action, afforded these 
schools professional development, instituted new research- based curriculum, 
developed joint school improvement plans, and notified parents about the status of 
the school. A modest number of districts appointed outside experts to advise the 
schools, replaced the principals, extended the school day, or decreased the 
management authority of the school. Few districts contracted the schools to a private 
entity to operate, replaced all the staff, or turned over the schools to the state. 

• Restructuring. About 449 schools in the 36 districts were in NCLB restructuring. 
Districts most often provided these schools with additional technical assistance, 
professional development, and additional planning help. A modest number of districts 
implemented tougher sanctions, such as decreasing management authority at the 
school, replacing the principal or relevant staff, restructuring or reorganizing the 
school, or appointing an outside advisor. Most districts stayed away from more 
punitive sanctions, including reopening the schools as charter schools, contracting 
with a private entity to run the schools, or turning over the schools to the state. 
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• Achievement. The effect of NCLB sanctions on student achievement was unclear. 
Achievement in the nation's urban schools has increased over the last several years. 
Most urban districts indicated that their gains were the result of more systemic 
instructional reforms beyond those called for in NCLB, but it is conceivable that NCLB 
helped produce these gains. 

Where to Obtain 

http://www.cacs.orq/imaaes/Publications/NCLB Fivevears.pdf 
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Education Commission of the States, 2005 

State involvement in school restructuring under No Child Left Behind in the 2004-2005 

school year 



Focus 

Examines how state education agencies have been involved in planning and implementing 
school restructuring under NCLB. Also gives a brief snapshot of how each of the 13 states 
studied had interpreted its role in restructuring. 

Methodology 

Based on interviews with state officials and reviews of relevant documents in the 13 states 
with schools in the implementation phase of NCLB restructuring in 2004-05. 

Major Findings 

• Different levels of state involvement. In 2004-05, the 13 states surveyed varied 
in their level of involvement in the school restructuring process. Seven of these 
states had approval processes for all school restructuring plans, two collected plans 
but did not officially approve them, one collected plans of only some schools, and 
three did not collect plans at all. 

• School choices. Most schools chose mild or moderate restructuring strategies— such 
as modifying curriculum, altering the school management structure, or choosing a 
school reform model— that fall under the federal option of "any other" major 
restructuring of school governance. The smaller number of districts that tried 
stronger interventions found themselves in the midst of difficult political battles. For 
example, the option of replacing staff sometimes conflicted with existing union 
contracts, while the charter school or private management options were often seen 
as politically unpalatable. However, some schools and districts used NCLB 
restructuring to replace staff who otherwise would have been difficult to replace. 

• Federal and state alignment. Alignment between NCLB and prior state 
accountability systems was difficult. States with robust accountability systems in 
place before NCLB had a difficult time combining the two systems. States that 
allowed significant interventions under their own accountability systems, such as 
conversion to a charter school or private management, rarely used these actions 
with schools that had to be restructured under NCLB. 

• Timing issues. Removing staff or reopening with different leadership was difficult 
when schools were not identified for improvement until after the school year began. 
States also complained that reforms made in the corrective action phase of NCLB 
were not given enough time to succeed before the restructuring phase kicked in. 

• Little federal guidance. All 13 states surveyed reported that the U.S. Department 
of Education had given them either no or insufficient guidance on how to implement 
restructuring. States generally viewed this as a negative. 

Where to Obtain 

http://www.ecs.orq/clearinqhouse/64/28/6428.pdf 
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Government Accountability Office, 2008 

No Child Left Behind Act: Education actions could improve the targeting of school 
improvement funds to schools most in need of assistance 



Focus 

Examines how Title I school improvement dollars are allocated and expended and the types 

of improvement activities that schools use and find effective. 

Methodology 

Administered a survey to state education agency officials in all 50 states and the District of 

Columbia between July and October 2007. Interviewed officials in each state and in 12 

districts and 22 schools. 

Major Findings 

• Source of school improvement funds. Since NCLB was enacted, 22 states have 
been unable, for one or more years, to set aside the full 4% of Title I funds for 
school improvement due to the law's hold harmless provision. In addition to Title I 
school improvement funds, 38 states have dedicated other federal funds and 17 have 
contributed state funds to school improvement efforts. 

• Allocation of school improvement funds. Though states generally targeted 
improvement funds to the most persistently underperforming schools, some states 
did not fulfill all NCLB requirements for allocating or tracking these funds. To allocate 
school improvement funds, 37 states used state-established criteria, which included 
factors such as the number of years schools have been identified as needing 
improvement. Two states used a competitive grant process, and eight used another 
method. Four states reported, however, that they required funds be allocated equally 
to schools and may not have considered factors required by NCLB, such as focusing 
on the lowest-achieving schools. The U.S. Department of Education has not provided 
guidance on how states should make publicly available lists of schools receiving 
funds and does not monitor states' compliance with this requirement. 

• I mprovement activities. Schools and states that received funds have undertaken a 
variety of improvement activities. For school year 2006-07, 45 states reported that 
schools which received improvement funds were engaging in professional 
development, reorganizing curriculum or instructional time, or analyzing data using 
student assessment information. To assess school improvement activities, 42 states 
reported that they analyze student achievement data or track school performance 
trends, and 36 of those states also obtain feedback from school and district officials. 

• ED support. ED provided various forms of support concerning school improvement. 
Department staff provided direct assistance with school improvement to states 
through written guidance, policy letters, and national meetings. ED's comprehensive 
centers provided technical assistance and research results to states on developing 
approaches for improving schools. ED made available information on school 
improvement strategies through its What Works Clearinghouse. 

Where to Obtain 

http: // www.aao.aov/ new. items/d08380. pdf 
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Government Accountability Office, 2007 

No Child Left Behind: Education should clarify guidance and address potential compliance 
issues for schools in corrective action and restructuring status 



Focus 

Examines the characteristics of Title I schools in corrective action and restructuring, the 
actions these schools implemented, the assistance they received from districts and states, 
and how the U.S. Department of Education supported state efforts to assist these schools. 

Methodology 

Administered two Web- based surveys to a nationwide sample of schools in corrective action 
and restructuring in 2005-06. Conducted site visits to five states with sizeable numbers of 
schools in corrective action or restructuring (California, Illinois, Maryland, New York, and 
Pennsylvania) and to 10 school districts and 20 schools in those states. Also interviewed 
state officials in four states (Idaho, South Dakota, Texas, and Virginia) with few schools in 
corrective action or restructuring. 

Major Findings 

• School characteristics. Nationwide, the 2,790 Title I schools that were in 
corrective action or restructuring status in school year 2005-06 were more often 
located in urban areas and in a few states. These schools served higher percentages 
of minority, poor, and middle-school students than other Title I schools, and many 
reported that factors like neighborhood violence and student mobility posed 
additional challenges to improving students' academic performance. 

• I mplementation. A majority of schools in corrective action or restructuring 
implemented the activities required by NCLB. GAO estimated, however, that 6% of 
schools did not take any of the required corrective actions, and about one-third of 
schools continued actions implemented during earlier years of school improvement 
rather than taking a new action after entering corrective action. In addition, about 
40% of schools did not undertake any of the five restructuring options in NCLB. ED 
does not require states to report on the specific measures taken for each school. 

• Assistance to schools. GAO estimated that 42% of schools in corrective action or 
restructuring did not receive all required types of assistance through their school 
districts, although most received discretionary assistance from their state education 
agencies. The U.S. Department of Education provided technical assistance and 
research results to states primarily through its Comprehensive Centers Program. ED 
also provided other material in its Web- based clearinghouse and was developing an 
initiative to outline practical steps for schools in various phases of improvement, 
including restructuring. 

Where to Obtain 

http: //WWW. aao. gov/ new. items/d071035. pdf 
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Learning Point Associates, 2006 

School restructuring under No Child Left Behind: What works when? 



Focus 

Aims to help education leaders choose the best restructuring options for schools with large 
numbers of students who are failing. Focuses on the four major restructuring options in 
NCLB law: reopening the school as a charter school, replacing the principal and staff, 
contracting with an outside entity to run the school, and turning over school operations to 
the state education agency. Includes practical decision-making tools and process steps 
based on research about restructuring. Considers strengths and constraints of implementing 
these restructuring options in a wide variety of school districts. 

Methodology 

Reviewed research on restructuring options under NCLB. Conducted interviews with 
educators and leading researchers with knowledge of when restructuring options do and do 
not lead to dramatic improvements in learning. Conducted a review of research literature 
that compared high-performing schools to other schools with similar populations. 

Major Findings 

Education leaders can use four main steps to choose the best restructuring options for their 
school. (The report also describes the detailed actions involved in each step.) 

Step 1. Take charge of big change. I ncludes organizing the district restructuring 
team, assessing the team's and district's capacity to govern restructuring decisions, 
deciding whether to invite a state takeover of the entire restructuring process, making a 
plan to include stakeholders, and preparing the district team to take further action. 

Step 2. Choose the right changes. Includes organizing the school- level 
decisionmaking process, conducting a school-by-school restructuring analysis, and 
making final restructuring decisions across the whole district. 

Step 3. Implement the plan. Includes setting goals for implementation and identifying 
and tackling likely roadblocks to success. Lists resources to help with full implementation 
of each restructuring strategy. 

Step 4. Evaluate, improve, and act on failures. Provides a brief list of actions 
needed to improve future restructuring efforts. 

Where to Obtain 

http://www.centerforcsri.orq/files/ RestructurinqGuide.pdf 
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Mass I nsight Education & Research I nstitute, 2007 

The turnaround challenge 



Focus 

Proposes a flexible, systematic approach to transforming schools deemed chronically 
underperforming under No Child Left Behind or state accountability systems. 

Methodology 

Conducted interviews with practitioners, researchers, leading policymakers, and reform 
experts in more than a dozen states, including extensive interviews with directors of school 
intervention in six major urban districts and with 50 school management and/or support 
organizations. Analyzed more than 300 reports, articles, and written resources. 

Major Findings 

• Need to rethink strategies. Efforts to turn around the lowest- performing 5% of 
schools have largely failed. A few large urban districts have undertaken promising 
turnaround strategies, but most are in their early stages. Chronically low-performing 
schools and the systems supporting them require fundamental rethinking. 

• State roles. To implement broadly the lessons learned from turnaround pioneers, 
states need to: a) require failing schools and their districts to pursue more proactive 
turnaround strategies or lose control over the school; b) make fundamental changes 
in the conditions in which these schools operate; c) develop a local marketplace of 
partners and providers skilled in turnaround efforts; and d) appropriate the $250,000 
to $1,000,000 per year needed to turn around a failing school. 

• Three C's. Three basic elements are required for a successful turnaround strategy: 

> Change conditions by creating a "protected space" free of bureaucratic 
restrictions and overly stringent collective bargaining agreements. Provide 
incentives to motivate people to do their best work. 

> Increase capacity among school staffs and leaders; develop a strong marketplace 
of local partners with the experience and ability to lead turnaround efforts. 

> Organize clusters of schools, either within or across districts, with their own lead 
turnaround partner providing comprehensive services. 

• Readiness dimensions. Turnaround efforts should emulate the following 
characteristics of high-performing, high- poverty schools: 

> Readiness to learn: Schools directly address student deficits with such strategies 
as an extended school day and longer year, action against poverty- related 
adversity, discipline and engagement, and close student-adult relationships. 

> Readiness to teach: Staff share responsibility for student achievement and stress 
collaboration and continuous improvement, instruction is personalized. 

> Readiness to act: Leaders make mission-driven decisions about people, time, 
money, and programs, and are adept at securing additional resources and 
leveraging partner relationships. Schools respond creatively to constant unrest. 

Where to Obtain 

http://www.massinsiqht.orq/resourcefiles/TheTurnaroundChallenqe 2007.pdf 
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RAND Education, 2006 

School and district improvement efforts in response to the No Child Left Behind Act 

Focus 

Addresses the strategies schools and districts are using to improve student performance, 
the perceived quality and usefulness of these efforts, and the perceived constraints and 
enablers of improvement efforts. 

Methodology 

Carried out case studies in school years 2003-04 and 2004-05 in three states (California, 
Georgia, and Pennsylvania). Data were drawn from interviews with 43 superintendents and 
surveys of 67 superintendents; surveys of 148 elementary principals and 112 middle school 
principals; and surveys of 1,833 elementary teachers and 1,340 middle school teachers. 

Major Findings 

• Strategies. School improvement strategies were well underway in all three states. 
Three of the most important improvement strategies identified by principals and 
superintendents were making increased use of achievement data to inform 
instruction, matching curriculum and instruction with standards and/or assessments, 
and providing additional instruction to low-performing students. 

• Data use. Districts and schools used a variety of student assessment data. 
Superintendents and principals found state test data especially useful for making 
decisions about improvement plans, focusing professional development, and making 
curriculum and instructional changes. Principals and teachers varied in their views of 
the usefulness of state assessment data, and many expressed concerns about the 
timeliness of the data. I n two of the three states, teachers found the results of 
progress tests administered throughout the year more helpful than state test results. 

• Alignment. Almost all principals reported aligning curriculum and instruction with 
state standards and/or assessments. Districts actively supported these efforts by 
providing pacing schedules, calendars, sample lessons, classroom feedback, and 
curriculum mapping. Although more than half of teachers in all three states viewed 
these district alignment activities as helpful, some expressed concerns about pacing 
and lack of time to teach for understanding and mastery. 

• Help for low-performing students. In all three states, about half or more of the 
districts studied required some or all schools to provide remedial instruction outside 
of the school day. In Georgia and Pennsylvania, many districts required schools to 
spend more time on math for low-achieving students. Other popular strategies 
included creating separate classes for low- performers and eliminating some remedial 
math classes in favor of more challenging instruction. 

• Obstacles. The number one hindrance to school improvement reported by 
superintendents and principals was inadequate funding, although case study data 
suggested that state and district capacity issues may also be due to a lack of 
qualified, trained personnel. Other perceived hindrances included frequent changes 

in state policy or leadership, teacher association rules in unionized states, inadequate 
lead time to prepare for reforms, and a lack of information on effective, research- 
based teaching methods and on ways to tailor instruction for students with 
disabilities and English language learners. 
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Where to Obtain 

http://www.rand.orq/pubs/workinq_papers/2006/ RAND_WR382.pdf 



CEP, Compendium of Major NCLB Studies School Improvement & Restructuring 



Page 16 



U.S. Department of Education, Institute of Education Sciences, 2008 

Turning around chronically low-performing schools: A practice guide 



Focus 

Identifies evidence- based recommendations that educators can use to quickly and 
dramatically improve student achievement in chronically low-performing schools. 

Methodology 

Based on 10 case studies of 35 low-performing "turnaround" schools that improved student 
achievement in one to three years; expert analyses of turnaround practices; and correlation 
studies and longitudinal studies of patterns of school improvement. However, none of the 
studies analyzed were based on a research methodology that yielded valid causal inference. 

Major Findings 

The study makes four recommendations and provides details on the steps needed to follow 
the recommendations: 

1. Signal the need for dramatic change with strong leadership. Schools should 
make a clear commitment to dramatic changes from the status quo, and the leader 
should signal the magnitude and urgency of that change. 

2. Maintain a consistent focus on improving instruction. Chronically low- 
performing schools need to maintain a sharp focus on improving instruction at every 
step of the reform process. To improve instruction, schools should use data to set 
goals for instructional improvement, make changes to immediately and directly affect 
instruction, and continually reassess student learning and instructional practices to 
refocus the goals. 

3. Make visible improvements early in the school turnaround process. Quick 
wins can rally staff around the effort and overcome resistance and inertia. 

4. Build a committed staff. The school leader must build a staff that is committed to 
the school's improvement goals and qualified to carry out school improvement. This 
goal may require changes in staff, such as releasing, replacing, or redeploying staff 
who are not fully committed to turning around student performance and bringing in 
new staff who are committed. 

Where to Obtain 

http://ies.ed.qov/ncee/wwc/pdf/practiceauides/Turnaround pq 04181.pdf 
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U.S. Department of Education, Policy and Program Studies Service, 2007 

National Assessment of Title I: Final report, vol. I 



Focus 

Comprehensive study that describes the progress of states, districts, and schools through 
school year 2004-05 in implementing key provisions of Title I, including assessment and 
accountability provisions. Summarized below are the study's findings about school 
improvement. 

Methodology 

Drew on data from a set of implementation studies by the U.S. Department of Education. 
Data for these studies came from surveys conducted in a nationally representative sample 
of school districts, other state and local surveys, state performance reports, and the 
National Assessment of Educational Progress. 

Major Findings 

• State notification of school improvement status. Only 15 states provided final 
notice to schools before September 2004 of their improvement status for 2004-05. 
Thirty states provided preliminary results by that time. 

• Statewide support for identified schools. Almost all states had implemented a 
statewide system of support for identified schools by fall 2004. These often involved 
school support teams (37 states) and individual school improvement specialists (29 
states). Most states (42) reported that providing assistance to all schools identified 
for improvement was a moderate or serious challenge in 2003-04. 

• Improvement strategies. Schools identified for improvement were more likely 
than non- identified schools to report needing assistance in a variety of areas; 80% 
reported needing assistance to improve the quality of teachers' professional 
development. The most common improvement strategies implemented by identified 
schools included developing a school improvement plan, using assessment data to 
inform instruction, and providing additional instruction to low-achieving students. 

• Corrective action. About 95% of Title I schools in corrective action in 2004-05 
underwent the interventions laid out by NCLB. The most common interventions for 
Title I schools in corrective action in 2003-04 and 2004-05 resembled forms of 
technical assistance rather than sanctions. For instance, 89% of these schools were 
required to implement new research- based curricula or instructional programs, and 
59% had an outside expert appointed to advise the school. Corrective actions were 
also often implemented in schools at other stages of NCLB improvement. 

• Parent information. Nearly one-quarter of principals and teachers in identified 
schools were unaware that their school had been identified for improvement. Parents 
in eight urban school districts were much less likely than principals or teachers to 
know whether their child's school had been identified as low-performing. 

Where to Obtain 

http://ies.ed.qov/ncee/pdf/20084014 rev. pdf 
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Center on Education Policy, 2007 

State implementation of supplemental educational services under NCLB 



Focus 

Examines efforts states have undertaken to carry out the supplemental educational services 
(tutoring) requirements of NCLB, focusing on school year 2006-07. Describes the 
procedures used to review and approve potential SES providers and the extent to which 
states were able to monitor the quality and effectiveness of SES providers. 

Methodology 

Draws data from a fall 2006 survey of state education agency officials in 50 states. 

Major Findings 

• State capacity to monitor. Thirty-eight states reported being unable to monitor 
the quality and effectiveness of SES providers "to a great extent." Only 10 states 
reported being able to do so "to a great extent." States attributed their inability to 
monitor to insufficient staff and funds. 

• Use of criteria in law. Between 47 and 49 states reported using the criteria 
required by federal law and guidance to review and approve applications from 
potential SES providers. These criteria are intended to ensure that providers are 
financially sound, have a record of effectiveness, use research- based strategies, 
provide services consistent with district instruction, and adhere to health, safety, 
and civil rights laws. 

• Frequent updating. Twenty states said they reviewed new SES provider 
applications more than once a year (the minimum required by the NCLB law); 22 
states reported updating their provider lists more than once a year. 

• Different reapplication policies. Each state has a different reapplication process 
for providers. Thirteen never require a formal reapplication, while 12 states require 
providers to reapply every year. 

Where to Obtain 

http: //www.cep-dc.orq 
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Center on Education Policy, 2006 

From the capital to the classroom: Year 4 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the implementation and impact of various NCLB 
provisions during school years 2004-05 and 2005-06. Summarized below are the study's 
findings about public school choice and supplemental educational services. 

Methodology 

Collected data through a survey of all 50 states, a nationally representative survey of 299 
school districts, case studies of 38 geographically diverse districts and 42 schools, three 
national forums, and six special analyses of critical issues in implementing NCLB. 

Major Findings 

• Percentage eligible for SES. I n 2005-06, 12% of districts were required to offer 
SES, and 15% of students in those districts were eligible for these services. These 
percentages were similar to those in the preceding three years. A larger proportion 
of urban districts (40%) had schools required to offer supplemental services than did 
suburban districts (12%) or rural districts (9%). 

• Participation in choice and SES. More eligible students used supplemental 
educational services than used the NCLB choice option. The percentage of eligible 
students who participated in SES in 2005-06 was 20%, about the same as in the 
preceding year. The percentage of eligible students who changed public schools 
under the NCLB choice option was less than 2%. 

• Providers. The average number of supplemental service providers per district grew 
dramatically between 2002-03 and 2004-05, from 4 providers to 20. States reported 
that as of August 2005 more than half of their providers (54%) were for-profit 
entities, while 21% were nonprofit entities, and 9% were school districts. The 
percentage of urban districts that were approved providers declined significantly 
from 43% in 2003-04 to 13% in 2005-06. A similar drop occurred among suburban 
districts. This decline may have occurred because some urban and suburban districts 
were identified for improvement and were no longer allowed to directly provide SES 
except in special cases. 

• Challenges. The greatest challenges to implementing supplemental services related 
to monitoring the quality and effectiveness of SES providers. Forty-one states and 
about half (51%) of school districts called this a moderate or serious challenge. 

Where to Obtain 

http: //www.cep-dc.orq 
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Center on Education Policy, 2005 

From the capital to the classroom: Year 3 of the No Child Left Behind Act 



Focus 

Comprehensives study that describes the implementation and impact of various NCLB 
provisions during school years 2003-04 and 2004-05. Summarized below are the study's 
findings about public school choice and supplemental educational services. 

Methodology 

Collected data through a survey of 49 states, a nationally representative survey of 314 
school districts, case studies of 36 geographically diverse districts and 37 schools, three 
national forums, and four special analyses of critical issues in implementing NCLB. 

Major Findings 

• Implementation of choice. Although about 15% of districts had schools that were 
required by NCLB to offer public school choice in 2004-05, very few students— a 
miniscule 1% of those eligible— took advantage of this option. The choice 
requirement disproportionately affected large districts and urban districts. In 2004- 
05, about 48% of urban districts had schools that were required to offer choice. 

About a third of all Title I school districts reported having moderate to serious 
problems finding physical space for students who wanted to transfer schools and 
sticking to class size limits in receiving schools. 

• Impact on achievement. Only 3% of the school districts required to offer choice 
said they believed the choice option was improving student achievement; 28% of 
these districts said choice was having a minimal effect or no effect on achievement; 
and 69% did not know what impact choice was having on achievement. 

• SES participation. In 2004-05, about 10% of Title I districts had schools that were 
required to offer supplemental educational services. Only 18% of students eligible for 
these services were actually participating in them. Although 42% of school districts 
said they did not know what effect supplemental services were having on student 
achievement, 20% said they believed these services were raising student 
achievement at least somewhat. 

• SES providers. Private, for-profit companies constituted about half the approved 
SES providers in 2004-05. School districts comprised 26% of approved providers, 
down from 37% the previous year, probably because federal regulations prohibited 
districts identified for improvement from directly providing SES. States and school 
districts voiced concerns about the lack of sufficient oversight of outside tutoring 
providers. Roughly three- fourths of the states reported that monitoring the 
effectiveness and quality of SES providers was a serious or moderate challenge. 

Where to Obtain 

http: //www.cep-dc.orq 
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The Civil Rights Project, UCLA, 2007 

Supplemental educational services under NCLB: Charting implementation 



Focus 

Examined trends in implementation of supplemental educational services, student 
participation in SES, and benefits of SES from 2002-03 to 2006-07. 

Methodology 

Collected data from six states' Web sites (Arizona, California, Georgia, Illinois, New York, 
and Virginia). Also collected data directly from officials in 11 districts in these states with 
large minority and low-income enrollments, including the nation's three largest districts. 

Major Findings 

• SES eligibility. The number of students eligible for SES and the absolute number of 
students receiving services increased over five years. These increases were related 
to a rise in the number of schools identified for improvement and required to offer 
SES. Among the districts studied, the percentage of students eligible for SES in 
2006-07 varied, ranging from over 5% in Richmond to more than 50% in Chicago 
and Fresno. 

• SES participation. The percentage of eligible students actually receiving services 
declined or leveled off after five years. The low demand for SES continued even 
though the Administration exerted strong pressure on districts to expand the 
program. 

• Providers. Over five years, the number of SES providers expanded greatly in four of 
the six states studied, stayed about the same in one state, and declined in one state. 
The number of providers grew substantially in all 11 districts. 

• Minority enrollments in SES schools. Schools that were identified for 
improvement, and therefore potentially subject to the SES requirement, enrolled 
higher proportions of African American and Latino students than schools that made 
adequate yearly progress. 

• Lack of evidence of impact. Five years after NCLB mandated SES, there was very 
little evidence documenting the effectiveness of SES in improving student 
performance. 

Where to Obtain 

http://www.civilriqhtsproiect.ucla.edu/research/esea/SES Policy Brief.pdf 
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Council of the Great City Schools, 2006 

No Child Left Behind in America's Great City Schools: Five years and counting 



Focus 

Comprehensive study that examines the status and implementation of NCLB in the large 
urban districts that comprise the Council of the Great City Schools. Summarized below are 
the study's findings about the public school choice and supplemental services requirements. 

Methodology 

Based on data from 36 large urban districts, enrolling more than 5.1 million students, that 
responded to a survey by the Council of Great City Schools of its 66 member districts. The 
survey focused on NCLB implementation from school years 2002-03 through 2005-06. 

Major Findings 

• Choice participation. In the 36 responding districts, a relatively small number of 
students— about 1.7% of those eligible— transferred under the NCLB school choice 
provisions in 2005-06. Even so, this represented an increase in the number of 
compared with 2002-03. Districts appeared to be giving parents more options for 
receiving schools than they initially had; the norm was to give parents four or more 
choices of transfer schools at the elementary level, up from two or three in 2002-03. 
The reasons for low participation appeared to be related to capacity, information, 
and demand. 

• SES participation. The 34 districts in the survey with schools required to offer SES 
reported that 180,730 students, or almost 16% of those eligible, participated in SES 
in 2005-06. This was about the same rate of eligible students served as in 2003-04. 

• SES providers. About 95% of the SES participants in the responding districts were 
served by a private provider. Most of these urban districts could not be direct 
providers of SES because they had been identified for improvement. The urban 
districts that were allowed to offer their own services had higher SES participation 
rates. The number of external providers remained high, but areas with a shortage of 
providers included services for ELLs, students with disabilities, and middle and high 
school students. 

• Parent communication. The urban districts participating in the study provided 
parents with somewhat longer windows in which to make decisions about NCLB 
choice and SES than in previous years. The districts also attempted to communicate 
with parents through a wider variety of methods than was initially the case. In 
addition, districts appeared to notify parents of their options somewhat earlier and in 
multiple languages. However, letters to parents could have been clearer. 

• Challenges to choice. Urban districts indicated that their biggest challenge in 
boosting participation in choice was rooted in the timing of the various choices, 
including timelines for receiving data from the state on the improvement status of 
schools. Other challenges to implementing choice included parent communication, 
limited numbers of higher- performing schools to receive transfer students, and 
available space in schools. 

• SES challenges. The timing of notification about the improvement status of schools 
continued to be a problem in organizing SES in school year 2005-06. Other problems 
reported by districts included getting students to show up for services, aligning 
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providers' curriculum and instruction with that of the district, evaluating SES offered 
by private providers, and negotiating contracts. 

• Cost. Overall, the 36 districts studied budgeted the equivalent of 17.2% of their Title 
I allocations for choice and SES in 2005-06. The cost of SES offered by external 
providers ran two to three times as much as the cost for district or school providers. 

Where to Obtain 

http://www.cacs.orq/imaqes/Publications/NCLB Fivevears.pdf 
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Education and the Public I nterest Center (University of Colorado-Boulder) & 
Education Policy Research Unit (Arizona State University), 2007 

Supplemental educational services under NCLB: Emerging evidence and policy issues 

Focus 

Analyzed student eligibility and participation in supplemental educational services, state and 

district implementation of SES, and the impact of SES on student achievement. 

Methodology 

Synthesis of evaluation studies of at least 17 providers, studies of Minneapolis and Chicago 

Public Schools, survey data on up to 49 states, and other academic and scholarly research. 

Major Findings 

• Participation. SES eligibility rates were much higher than enrollment rates. Most 
estimates suggested that 15% to 20% or fewer of all eligible students receive SES. 
Limited services are available for English language learners and students with 
disabilities. Little is known about which eligible students are not participating. 

• District implementation. Evidence is mixed about how well districts are performing 
their responsibilities for administering and implementing SES. Some studies 
suggested that districts are making a good faith effort to implement SES— for 
example, by hiring extra staff to administer the program and making aggressive 
efforts to notify and enroll students, track attendance, and monitor providers. Other 
research contended that districts fell short on parent outreach and notification. 

• Capacity to monitor. Districts and states had limited capacity to carry out their SES 
responsibilities and monitor program quality due to underfunding, funding "set aside" 
limitations, and their dependence on SES providers to make information available. 

• Impact on achievement. While studies on the impact of SES on student 
achievement were virtually non-existent, the Minneapolis Public Schools found that 
the average growth for SES students was only 66% of the national norm. The 
Chicago Public Schools found that students who received at least 40 total hours of 
tutoring had higher math and reading gains than those who received fewer than 40 
hours, and that students who received SES from seven providers (including the 
district itself) had higher reading scores than the district average. 

• Conditions for positive outcomes. Existing research offers little information about 
the specific conditions that support positive student outcomes. 

Where to Obtain 

http://epicpolicv.orq/files/EPSL-0705-232-EPRU.pdf 
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Government Accountability Office, 2006 

No Child Left Behind Act: Education actions needed to improve local implementation and 
state evaluation of supplemental educational services 



Focus 

Examines early implementation of supplemental educational services, including changes in 
participation, relationships of providers and districts, and state and federal monitoring, 
evaluation, and support. 

Methodology 

Drew from survey of all states; a nationally representative sample of 21 districts from the 
1,000 districts with schools required to offer SES; visits to 4 districts; and interviews with 
22 SES providers. 

Major Findings 

• SES participation. Participation in SES increased from 12% to 19% between school 
years 2003-04 and 2004-05, partly due to a rise in the number of schools required 
to offer SES. 

• Notifying parents and improving delivery. Districts used some promising 
practices to inform parents and encourage participation, such as offering services at 
school sites and at various times. Challenges remained, including notifying parents 
effectively and promptly, finding SES providers to serve rural areas and students 
with disabilities, and contracting and coordinating service delivery. Greater 
involvement of schools could improve SES delivery. 

• Coordinating SES with school curricula. SES providers took steps to 
communicate with teachers and parents and to align curriculum with district 
instruction, through such means as hiring teachers familiar with district curriculum as 
tutors. The degree of these efforts varied; some providers did not have any contact 
with teachers in about 40% of districts or with parents in 30% of districts. 

• Monitoring. During 2005-06, states reported conducting more on-site reviews and 
other monitoring of SES providers than in past years. Districts also increased their 
monitoring. U. S. Department of Education monitoring found uneven implementation 
and compliance with SES provisions. States and districts reported needing additional 
evaluation support and technical assistance.* 

• Evaluating effects. States struggled with how to evaluate providers' impact on 
achievement; 85% of states desired more assistance from ED about evaluation 
methods. No state evaluation provided a conclusive assessment of providers' effect 
on student achievement. 

Where to Obtain 

http://www.aao.qov/new.items/d06758.pdf 



‘According to GAO testimony before Congress on Aprii 18, 2007 ( http://www.aao.aov/new.items/d07738t.pdf ). ED 
has since taken actions to improve SES impiementation and monitoring, such as disseminating promising practices 
and guidance, and meeting with states, districts, and providers. 
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University of Wisconsin- Madison, 2007 

Supplemental educational services and NCLB: Policy assumptions, market practices, 
emerging issues, Educational Evaluation and Policy Analysis 



Focus 

Examined market dynamics in relationship to NCLB's goals of expanding access to and 
improving the quality of after-school programming. 

Methodology 

Analyzed operational and financial data from seven SES providers, case study data on SES 
provider activity (from 2004 to 2006) in an urban district, and survey data from 30 state 
administrators. 

Major Findings 

• Effect of market forces. I n the early years of implementation, the SES market 
exhibited anticompetitive forces. There may have been low barriers for private 
providers to enter the market, as reflected in the exponential increase in the number 
of approved providers nationally. However, a small percentage of providers were 
positioned to capture most of the market, as reflected in the size of their revenue 
increases, merger and acquisition activity, and interstate marketing strategies. 

• Firms with market share. The SES firms that gained market share in a large urban 
school district from 2004 to 2006 charged higher hourly rates on average and had 
larger class sizes. Although these firms garnered significant revenues from providing 
SES, they did not offer services accessible to students with English language learners 
and students with disabilities. 

• Limited monitoring capacity. A survey of 30 state Title I administrators revealed 
the limited capacity of state agencies to monitor industry leaders in ways that hold 
them accountable for meeting students' needs. 

• Mismatch of implementation and federal priorities. A mismatch existed 
between the implementation dynamics of SES and the concerns for quality, access, 
and equity that are the stated priorities of NCLB law. 

Where to Obtain 

http://www.education.wisc.edu/eps/Facultv%20papers/EEPA302035_Revl.pdf 
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U. S. Department of Education, Policy and Program Studies Service, 2008 

State and local implementation of the No Child Left Behind Act. Volume IV— Title I school 
choice and supplemental educational services: Interim report 



Focus 

Examines implementation of Title I school choice and supplemental educational services 
provisions during school years 2003-04 and 2004-05, as part of ED's National Longitudinal 
Study of NCLB. 

Methodology 

State- level interviews and surveys of nationally representative samples of district officials, 
principals, and teachers. Surveys of parents in 8 districts, SES providers in 16 districts, and 
student- level demographic and achievement data in 9 districts. 

Major Findings 

• District and school implementation. Most districts and schools that were required 
to provide Title I school choice and SES did so. Elementary schools were more likely 
than middle or high schools to offer these options. Most districts (63%) offered 
parents at least three options for SES, and 38% of districts offered parents five or 
more options. 

• Participation. Only a small proportion of eligible students participated in Title I 
school choice and SES in 2003-04— about 17% of those eligible for SES, and about 
1% of those eligible for choice. Reasons for this lack of participation include an 
absence of available options, timing of parent notification, and problems 
communicating with parents. 

• Characteristics of participants. On average, students who participated in Title I 
choice and SES had lower prior achievement than other students in their district or 
than students who were eligible but chose not to participate. Eligible students were 
more likely to be minorities than those who did not qualify. In the nine districts 
surveyed, African American students were the most likely racial-ethnic group to 
participate. 

Where to Obtain 

http://www.ed.qov/ rschstat/eval/choice/ncib-choice-ses/ high lights, pdf 
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U.S. Department of Education, Institute of Education Sciences, 2007 

National assessment of Title I final report. Volume 1: Implementation 



Focus 

Comprehensive study that evaluated implementation of key Title I provisions related to 
participants' characteristics, targeting and use of funds, state assessments, accountability 
and school improvement, school choice and supplemental educational services, and teacher 
quality; also examined trends in student achievement. Summarized below are findings 
about choice and supplemental educational services. 

Methodology 

Draws on data from the National Longitudinal Study of NCLB, which included surveys of 
districts, principals, classroom teachers, special education teachers, and Title I 
paraprofessionals in a nationally representative sample of 300 districts and 1,483 schools in 
school years 2004-05 and 2006-07. The study also surveyed parents in eight districts and 
supplemental service providers in 16 districts in the same two years, and analyzed 
achievement outcomes for students participating in choice and SES options in nine districts. 
Additional information was drawn from case studies in nine districts that looked at early 
experiences with implementing SES in 2002-03 and 2003-04. 

Major Findings 

• Eligibility and participation in choice and SES. Although twice as many students 
were eligible for Title I school choice as were eligible for SES (5.2 million versus 2.4 
million), nearly ten times as many students actually participated in SES (446,000 in 
SES versus 48,000 in choice). Student participation in the choice option doubled 
from 2002-03 to 2004-05, while participation in SES increased more than tenfold. 
Choice participation rates varied widely across states and districts; of the districts 
required to offer choice in 2004-05, 63% reported that no students participated that 
year, while other reported higher participation rates. 

• Characteristics of participants. In the nine urban case study districts, students 
participating in choice and SES were more likely than other students in their districts 
to be minority students and have below-average achievement. School choice 
participants typically transferred from a school with below-average achievement to a 
school with above-average achievement and chose schools with lower concentrations 
of minority students than they schools they left. 

• Costs. In 2003-04, districts spent an estimated $24 million on transportation for 
Title I choice participation and an estimated $192 million for SES. Districts reported 
spending an average of $875 per participating student for SES in 2003-04, about 
30% less than the maximum per child that they had allocated that year. 

• Schools offering SES and choice. The number of schools offering SES tripled from 
800 in 2002-03 to 2,500 in 2003-04. The number offering Title I choice increased 
from 5,100 in 2002-03 to 6,200 in 2004-05. 

• Parent notification. Although nearly all districts required to offer choice and SES 
reported that they notified parents about these options, a parent survey in eight 
urban districts found that only 27% of parents whose children were eligible for choice 
and 53% of parents whose children were eligible for SES said they had been notified. 
The quality of district notification letters varied considerably; some were easy to read 
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and presented the options in a positive light, while others were confusing, 
discouraged the use of the options, or were biased in favor of district- provided 
services. Often parents were notified too late to choose a new school before the start 
of school year 2004-05. 

• Providers and services. Private firms accounted for 86% of providers in May 2007, 
while school districts and public schools accounted for only 11%. However, data from 
2003-04 indicted that the public providers served more students per provider. In the 
16 districts involved in a special survey, supplemental services were most commonly 
provided at the student's school. Services were provided for an average of 57 hours 
per year, and students attended about 78% of the sessions, on average. 

• Monitoring. As of early 2005, most states were still working to develop and 
implement systems for monitoring and evaluating the performance of SES providers. 
At that time, 15 states had not established any monitoring process, 25 states had 
not yet established any standards for evaluating provider effectiveness, and none 
had finalized their evaluation standards. 

Where to Obtain 

http://ies.ed.qov/ncee/pdf/20084012 rev. pdf 
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U. S. Department of Education, Office of Planning, Evaluation, and Policy 

Development, 2007 

State and local implementation of the No Child Left Behind Act. Volume I: Title I 

school choice, supplemental educational services, and student achievement. 

Focus 

Examined the characteristics of students participating in the Title I public school choice and 

SES options and the related impact on student achievement. Part of the National 

Longitudinal Study of NCLB. 

Methodology 

Student-level data was collected for 2004-05 from nine large urban school districts. 

Major Findings 

• Participation. Participation in both choice and SES was highest at the elementary 
level. In grades 2 to 5, 24% to 28% of eligible students participated in SES, 
compared with less than 5% in high school. For school choice, the average 
participation rates were between 0.6% and 1.0% of eligible students in grades 2-5, 
and between 0.2% and 0.4% in high school. 

• Student characteristics. African American students had the highest participation 
rate among racial-ethnic groups in SES and higher than average participation in 
school choice. Latino students had higher participation than white students in SES 
but lower participation in school choice. English language learners and students with 
disabilities had relatively high participation rates in SES and relatively low 
participation in school choice. 

• Prior achievement of participants. Among students eligible for SES, those who 
enrolled to receive these services had lower prior achievement levels than those who 
did not enroll. Among students eligible for Title I school choice, those who actually 
transferred schools had prior achievement levels similar to those who did not 
transfer. For both options, participating students had lower prior achievement levels 
than those of ineligible students. Overall, students who took advantage of school 
choice and SES generally came from disadvantaged populations. 

• Effect of SES on achievement. On average, across the seven study districts with a 
sufficient sample of students, SES participation had a statistically significant, positive 
effect on students' achievement in reading and math. Students participating for 
multiple years experienced larger gains. 

• Choice and achievement. Students who used Title I choice moved from schools 
with below-average achievement levels to schools with above-average levels, and 
tended to move to more racially balanced schools. In the six districts with a sufficient 
sample of students, no statistically significant effect on achievement was found for 
students who changed schools. Flowever, the sample sizes for choice were small, so 
results should be interpreted with caution. 

Where to Obtain 

http://www.rand.orq/pubs/reprints/2007/RAND RP1265.pdf 
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Center on Education Policy, 2007 

Implementing the No Child Left Behind teacher requirements 



Focus 

Examines how states and school districts have carried out the NCLB teacher requirements; 
part of a comprehensive, multiyear study of NCLB implementation. 

Methodology 

Collected data through an annual survey of 50 states; an annual, nationally representative 
survey of 349 responding school districts; case study interviews with local administrators in 
17 school districts; and two roundtables with education association representatives. 

Major Findings 

• Minimal impact on achievement. Fifty-six percent of states and 66% of districts 
responding to CEP's surveys reported that the NCLB highly qualified teacher 
requirements have had minimal or no impact on student achievement. (The highly 
qualified teacher requirements of NCLB essentially require teachers to demonstrate 
knowledge in the subjects they teach by holding a degree in their subject, 
completing more coursework, or other means.) Only 6% of states and 4% of districts 
indicated that the requirements have improved achievement to a great extent. 

• Impact on teacher effectiveness. The NCLB teacher quality requirements have 
had minimal or no impact on teacher effectiveness, according to 38% of states and 
74% of districts responding to CEP's surveys. Only 8% of states and 6% of districts 
said these requirements have improved teacher effectiveness to a great extent. 

• Compliance. As of late fall and winter of 2006-07, 66% of school districts reported 
they were in full compliance with the NCLB highly qualified teacher requirements, 
and 17% expected to achieve full compliance by the end of that school year. But 
only three states reported being in full compliance as of fall/winter 2006-07; 14 
more states expected to reach full compliance by the end of that school year. 
However, 22% of states and 6% of districts doubted they would ever meet the 
requirement for all of their teachers to be highly qualified according to the NCLB 
definition. 

• Groups presenting challenges. Eighty-three percent of states and 47% of districts 
reported having problems complying with the highly qualified requirements for 
special education teachers. Secondary school science and math were another group 
of teachers that presented a compliance problem for states and districts. 

• Recruitment strategies. Induction and mentoring programs and content-driven 
professional development were the most common recruitment and retention 
strategies districts reported using to recruit and retain highly qualified teachers. 

Other strategies used included enhanced outreach, course tuition assistance, 
assistance in preparation for state licensure, and certification exams. 

• Equitable distribution. Only five states reported that the distribution of 
experienced, well-qualified teachers in high-poverty and high-minority-enrollment 
schools had become more equitable "to a great extent," while 17 states said this 
distribution had become somewhat more equitable, and 17 other states reported 
minimal change. Among districts with more than one school, 55% reported that the 
distribution of experienced, well-qualified teachers has remained equitable or that no 
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real difference in teacher qualifications existed in their district based on their schools' 
poverty or minority enrollments. Still, some districts did face challenges in ensuring 
an equitable distribution of teachers to these schools. 

• "Highly qualified" definition. Many state and district officials thought the 
definition of a highly qualified teacher was too narrowly focused on content 
knowledge. Many suggested revising the definition to take into account teachers' 
effectiveness in the classroom and other qualities essential to a good teacher. 

Where to Obtain 

http: //www.cep-dc.orq 
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Center on Education Policy, 2006 

From the capital to the classroom: Year 4 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2004-05 and 2005-06. Summarized below 
are the study's findings about teacher requirements. 

Methodology 

Collected data through a survey of all 50 states, a nationally representative survey of 299 
school districts, case studies of 38 geographically diverse districts and 42 schools, three 
national forums, and six special analyses of critical issues in implementing NCLB. 

Major Findings 

• Teacher qualifications. At the time of the study in 2005-06, school districts were 
on their way to meeting the highly qualified teacher requirements of NCLB. Of the 
districts surveyed, 88% expected to meet the law's original deadline for all teachers 
of core academic subjects to be highly qualified by the end of school year 2005-06. 

• Skepticism about effects. Only 9% of state respondents and 8% of district 
respondents said they believed the NCLB teacher quality requirements have 
improved the quality of teaching to a great extent. Roughly a third of both state and 
district respondents said they believed the requirements have had some impact, but 
59% of district respondents said the requirements have had little or no impact. 

• Urban and rural differences. The proportion of districts that reported being on 
track to have all of their academic teachers highly qualified by the end of school year 
2005-06 was similarly high across urban, suburban, and rural districts. In contrast to 
previous years' data, CEP found no significant difference in the percentage of high- 
minority-enrollment districts and lower-minority-enrollment districts reporting that 
all their teachers were highly qualified. Still, some urban districts participating in 
CEP's case studies said they had trouble hiring and keeping highly qualified teachers. 

• Challenges for certain types of teachers. Despite overall progress, states and 
districts reported having difficulty in meeting the highly qualified requirements for 
some teachers, such as special education teachers, high school math and science 
teachers, or teachers in rural areas who teach multiple subjects. 

• NCLB paraprofessional requirements. More than 80% of school districts reported 
that their Title I paraprofessionals would meet the NCLB qualifications requirements 
by the end of school year 2005-06. According to CEP's case studies, most 
paraprofessionals who were not highly qualified met the criteria by passing a 
competency test rather than getting a degree. 

Where to Obtain 

http: //www.cep-dc.orq 
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Center on Education Policy, 2005 

From the capital to the classroom: Year 3 of the No Child Left Behind Act 



Focus 

Comprehensive study that describes the federal, state, and local implementation and impact 
of various provisions of NCLB during school years 2003-04 and 2004-05. Summarized below 
are the study's findings about teacher requirements. 

Methodology 

Collected data through a survey of 49 states, a nationally representative survey of 314 
school districts, case studies of 36 geographically diverse districts and 37 schools, three 
national forums, and four special analyses of critical issues in implementing NCLB. 

Major Findings 

• Highly qualified status and disparities. At the time of the study in 2004-05, most 
current teachers already met the NCLB criteria for being "highly qualified," according 
to the states and school districts surveyed. School districts with large numbers or 
percentages of poor and minority students had the largest proportions of teachers 
who were not highly qualified in NCLB terms. 

• Impediments to full compliance. States and districts reported problems with 
ensuring that special education teachers, middle school teachers, and teachers in 
rural areas met the law's requirements, even after the U.S. Department of Education 
granted additional flexibility in these areas in 2003-04. States also reported 
problems implementing the data systems necessary to track teacher qualifications. 

• Professional development. NCLB brought greater focus to districts' professional 
development efforts— for example, by encouraging the use of literacy "coaches" and 
school support teams. 

• Paraprofessionals. At the time of the study in 2004-05, most Title I 
paraprofessionals were already highly qualified as defined by NCLB, according to the 
states and school districts surveyed. Districts were using a variety of strategies to 
help Title I paraprofessionals become highly qualified— from providing study courses 
aimed at helping paraprofessionals pass competency tests to paying for 
paraprofessionals to take college courses— but significant challenges remained for 
some paraprofessionals. 

Where to Obtain 

http: //www.cep-dc.orq 
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Education Trust, 2006 

Missing the mark: An Education Trust analysis of teacher-equity plans 



Focus 

Describes the plans states were required to submit by July 2006 under NCLB to remedy 
inequities in the assignment of inexperienced, unqualified, or out-of-field teachers to schools 
serving poor and minority children. 

Methodology 

Examined teacher-equity plans from all states and D.C. submitted to the U.S. Department 
of Education. 

Major Findings 

• Insufficient information. Most states failed to follow ED instructions and failed to 
analyze inequities in a way that would tell the public whether children in poverty and 
children of color were getting a fair share of teaching talent. For example, only 10 
states appropriately analyzed whether minority students were taught 
disproportionately by teachers who were not highly qualified. Only four states looked 
at whether poor students were taught disproportionately by inexperienced teachers. 
Just three states looked at all four types of inequities required by ED. 

• Inadequate plans. Most states failed to propose strong plans for addressing 
inequities. For example, more than half the states asserted that they would comply 
with their equity obligations by focusing exclusively on compliance with the NCLB 
highly qualified teacher provisions, which would ignore inequality in the distribution 
of inexperienced teachers. Almost no states submitted equity plans that proposed 
meaningful, measurable goals for achieving fairness in the distribution of teacher 
talent. Three states had not analyzed their data or come up with any equity plans at 
all as of J uly 2006. 

• Flawed data. Some states used flawed data analysis to conclude that no inequities 
existed in teacher assignments. 

• State confusion. States seemed confused about what they were supposed to do to 
prepare the equity plans. Some of this can be attributed to a lack of attention from 
ED to the teacher-equity plan provisions of NCLB. 

• Overall conclusion. "[TJhe overwhelming majority of states should be required to 
start over, with clearer guidance and more assistance from the Department of 
Education, to get this process moving in the right direction." 

Where to Obtain 



http://www2.edtrust.orq/NR/ rdonlvres/5E2815C9-F765-4821-828F- 
66F4D156713A/0/TeacherEauitvPlans.pdf 
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Government Accountability Office, 2005 

No Child Left Behind Act: Improved accessibility to Education's information could help states 
further implement teacher qualification requirements 



Focus 

Examines the status of state efforts to meet the NCLB teacher requirements and the use of 

Title II teacher improvement funds in selected districts. Also looks at how the U.S. 

Department of Education monitors and assists states in implementing these requirements. 

Methodology 

Analyzed teacher qualifications data submitted to ED by 47 states, conducted site visits to 6 

diverse states, visited 11 high-need school districts across these states, and interviewed 

national experts and ED officials. The work was conducted in 2004-05. 

Major Findings 

• State efforts to comply. I n the 47 states with data, the majority of core academic 
classes were taught by teachers who met NCLB requirements during school year 
2003-04. States had improved their ability to track and report the percentage of core 
academic classes taught by highly qualified teachers, but the quality and precision of 
these data were limited. Five of the 6 states visited allowed veteran teachers to 
demonstrate subject matter competency through MOUSSE (high objective uniform 
state standard of evaluation). State and district officials anticipated challenges in 
meeting the highly qualified requirements for teachers of multiple subjects. 

• Uses of Title 1 1 funds. All 11 districts studied used Title 1 1 funds for professional 
development, and most used Title II funds to reduce class size. Some districts 
reported shifting funds away from class size reduction to initiatives designed to 
improve teachers' subject matter knowledge and instructional skills. The majority of 
districts indicated that NCLB had led to improvements in the kinds of professional 
development funded with Title II. 

• Targeting of Title II. All 11 districts reported considering student achievement data 
and targeting Title II funds to improve instruction in the subjects in which students 
were lagging behind. In the 11 districts studied, few efforts funded with Title II 
targeted specific groups of teachers, such as teachers in high-poverty schools. 

• Teacher funds. Title II funds constituted a small proportion of total funds districts 
could use for teacher improvement; all districts visited used several other sources to 
support these programs. 

• ED role. In monitoring the NCLB teacher requirements, ED found several areas of 
concern, such as states not ensuring that newly hired teachers met the 
requirements. ED's multiple types of assistance included professional development 
for teachers and technical assistance to state officials. Officials from most states and 
districts visited said they were unaware of some ED resources about the teacher 
requirements or had difficulty locating them on ED's Web site. 

Where to Obtain 

http: // www.aao.qov/ new. items/d0625. pdf 
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U. S. Department of Education, Office of Elementary and Secondary Education, 

2008 

Findings from the 2007-08 survey on the use of funds under Title 1 1 , Part A 



Focus 

Describes how school districts used federal funds for teacher quality activities under Title II, 

Part A in school year 2007-08. 

Methodology 

Conducted a survey in 2007-08, stratified by district size and level of poverty, of a 

nationally representative sample of 800 school districts. 

Major Findings 

• More funds for large and low-income districts. About 97% of all districts 
received Title 1 1 -A funding for school year 2007-08. Large districts (with 10,000 or 
more students) received 66% of the total Title 1 1 -A allocation; small districts (with 
fewer than 1,000 students) received 5% percent. The highest- poverty districts 
received 61% of the total allocation; the lowest- poverty districts received 8%. 

• Professional development and class size reduction. In 2007-08, half (50%) of 
Title 1 1 -A funds was used for professional development activities for teachers, 
paraprofessionals, and administrators, and 27% was used to hire highly qualified 
teachers to reduce class size. These figures represent an increase in funds for 
professional development (up from 27% in 2002-03) and a decrease for class size 
reduction (down from 57% in 2002-03). Viewed another way, about 70% of districts 
used Title 1 1 -A funds for teacher professional development, and 50% used the funds 
for class size reduction. 

• Differences in uses by district poverty. In 2007-08, the highest-poverty and 
lowest- poverty districts allocated more Title 1 1 -A funds for teacher professional 
development than for class size reduction, while the opposite was true for districts 
with medium-low and medium-high poverty. 

• Other uses. About 4% of Title 1 1 -A funds was spent on strategies to help schools 
recruit and retain highly qualified teachers, principals and specialists, such as 
scholarships, loan forgiveness, signing bonuses, or differential teacher pay. About 
6% was used to promote professional growth and reward quality teaching, such as 
mentoring, induction, or exemplary teacher programs. 

• Transferability. About 1% of Title 1 1 -A funds was combined with other federal 
program funds under the Rural Education Achievement Program flexibility provisions 
of NCLB, and 3% was transferred to another title of ESEA through the NCLB funding 
transferability provisions. Most commonly, Title 1 1 -A funds were transferred to Titles 

I and V. 

• Highly qualified teachers. Districts reported that 96% of the 3.2 million teachers 
who taught in the core academic content areas were highly qualified in 2007-08 
according to NCLB requirements. 

• Subjects for professional development. The majority of Title 1 1 -A funds for 
teacher professional development were allocated to the subject areas of science 
(50%), math (21%), and reading (16%). About 3% was allocated to other academic 
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subjects— most often fine arts and foreign languages. About 4% was spent on 
professional development in other non-academic topics, such as classroom 
management strategies, use of assessments, and curriculum development. 

• Participation in professional development. About 95% of teachers in the core 
academic content areas received professional development in 2007-08. 

• Modes of professional development. Common modes of professional 
development included full-day workshops during the school day (more than 4.7 
million teachers participated); daily learning team sessions (more than 1.9 million 
teachers); one-day workshops outside of the school day (more than 1.9 million 
teachers); and multi-day workshops (more than 1 million teachers). 

Where to Obtain 

http://www.ed.aov/proqrams/teacheraual/findinqs2008.doc 
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U.S. Department of Education, Policy and Program Studies Service, 2007 

National Assessment of Title I: Final report, vol. I 



Focus 

Comprehensive study that describes the progress of states, districts, and schools through 
school year 2004-05 in implementing key provisions of Title I. Summarized below are the 
study's findings about requirements for teachers, paraprofessionals, and professional 
development. 

Methodology 

Drew on data from a set of implementation studies by the U.S. Department of Education. 
Data for these studies came from surveys conducted in a nationally representative sample 
of school districts, other state and local surveys, state performance reports, and the 
National Assessment of Educational Progress. 

Major Findings 

• Tests for new teachers. Most states (41) were meeting the requirement to test 
new teachers' content knowledge by using the Praxis II subject assessments 
developed by the Educational Testing Service. States varied considerably in the 
scores they required teachers to obtain on these exams to be certified to teach or to 
be deemed highly qualified under NCLB. For example, on the Praxis II Mathematics 
Content Knowledge assessment, used by 35 states, 10 states set their cut scores 
below the 25‘^ percentile, while one state set its cut score at the 75'^^ percentile. 

• MOUSSE. As of November 2006, all states allowed veteran teachers to demonstrate 
their subject-matter competency through a high objective uniform state standard of 
evaluation (FIOUSSE). The most common type of FIOUSSE option allowed teachers to 
earn points toward highly qualified status retroactively for such things as successful 
completion of certain college courses or years of teaching experience. Four states 
allowed teachers to earn some points for evidence of improved student achievement. 

• Highly qualified status. According to state- reported data for 50 states, 91% of 
classes were taught by highly qualified teachers in 2004-05. Principal and teacher 
reports provided somewhat lower estimates of the percentage of classes taught by 
highly qualified teachers, but this is because a sizeable percentage did not know 
their highly qualified status. Special education teachers and secondary mathematics 
teachers were more likely to report that they were not highly qualified under NCLB 
than were general elementary teachers and secondary English teachers. 

• Unequal distribution of highly qualified teachers. Students in schools that were 
identified for NCLB improvement were more likely to be taught by teachers who said 
they were not highly qualified than were students in non- identified schools. Schools 
with high concentrations of poor and minority students had more teachers who were 
not highly qualified than other schools did. In high-poverty schools, for example, 5% 
of elementary teachers and 12% of secondary English and math teachers reported in 
2004-05 that they were not highly qualified under NCLB, compared with 1% in low- 
poverty elementary schools and 3% in low- poverty secondary schools. 

• Unequal distribution of experienced and out-of field teachers. Even among 
teachers who said they were highly qualified under NCLB, those in high-poverty 
schools had less experience and were more likely to be teaching out-of-field than 




CEP, Compendium of Major NCLB Studies 



Teacher Quality 



Page 10 



their peers in low- poverty schools. Twelve percent of highly qualified teachers in 
high- poverty schools had fewer than three years' teaching experience, compared 
with 5% of highly qualified teachers in low-poverty schools. Similarly, 41% of highly 
qualified secondary English and mathematics teachers in high-poverty schools had a 
degree in the field they taught, compared with 52% in low- poverty schools. 

• Professional development. Most teachers reported receiving some professional 
development in reading and mathematics content and instructional strategies, but 
less than one-quarter participated in such training for more than 24 hours over the 
2003-04 school year and summer. For example, 90% of elementary teachers 
participated in at least one hour of professional development focused on instructional 
strategies for teaching reading, but only 20% participated for more than 24 hours 
over the 2003-04 school year and summer. Teachers in high-poverty schools were 
more likely to participate in professional development focused on reading and 
mathematics than were teachers in low- poverty schools. For example, 53% of 
secondary English teachers in high- poverty schools reported participating in 
professional development focused on in-depth study of reading or English, compared 
with 36% of their colleagues in low-poverty schools. 

• Title I paraprofessionals. According to principal reports, 63% of Title I 
instructional aides had been determined to meet NCLB qualification requirements as 
of school year 2004-05. However, 87% of Title I instructional aides indicated that 
they had at least two years of college (or an associate's degree) or had passed a 
paraprofessional assessment. Nearly one-quarter (23%) of Title I instructional aides 
reported that, of the time that they spent tutoring or working with students in a 
classroom, a teacher was present only half or less of this time. 

• Decreased reliance on paraprofessionals. The share of Title I -funded district and 
school staff who were aides declined from 47% in 1997-98 to 32% in 2004-05, while 
the share who were teachers rose from 45% to 55%. 

Where to Obtain 

http://ies.ed.qov/ncee/pdf/20084014_rev.pdf 
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U.S. Department of Education, Policy and Program Studies Service, 2007 

State and local implementation of the No Child Left Behind Act. Volume II— Teacher quality 

under NCLB: interim report 



Focus 

Describes the progress states, districts, and schools had made implementing the teacher 
and paraprofessional qualification provisions of NCLB through 2004-05. 

Methodology 

Analyzed data from state documents; telephone interviews with state officials in 50 states, 
Puerto Rico, and D.C.; a survey of a nationally representative sample of 300 districts and 
1,483 elementary, middle, and high schools within those districts; and surveys of 4,772 
elementary teachers, 2,081 secondary English or language arts teachers, 1,938 secondary 
math teachers, 1,408 special education teachers, 950 Title I paraprofessionals, 1,483 
principals, and 300 district administrators. 

Main Findings 

• Compliance. By 2004-05, about three-quarters of teachers reported they were 
considered highly qualified under NCLB for the classes they taught. Nearly one- 
quarter did not know their status, and 4% reported they were not highly qualified. 

• State variation. State policies for meeting the NCLB highly qualified teacher 
requirements varied greatly, both in the passing scores that new teachers had to 
meet to demonstrate content knowledge on tests and the extent to which state 
MOUSSE policies gave more weight to teachers' prior years of experience than to 
more direct measures of content knowledge and teaching performance. 

• Teachers not highly qualified. The percentage of teachers who were not highly 
qualified under NCLB was higher for special education teachers, teachers of English 
language learners, and middle school teachers, as well as teachers in high-poverty 
and high-minority schools. Teachers in high-poverty schools had less experience and 
were less likely to have a degree in the subject they taught. 

• Professional development. Although nearly all teachers reported taking part in 
content-focused professional development related to teaching reading or 
mathematics, a relatively small proportion participated in such learning opportunities 
for an extended period of time. 

• Paraprofessionals. About two-thirds of instructional paraprofessionals were 
considered qualified under NCLB, but nearly one-third did not know their status or 
did not respond to the study questions. Most paraprofessionals reported working 
under the direct supervision of a teacher, but some Title I instructional 
paraprofessionals reported working with students on their own without close 
supervision from a teacher. 

Where to Obtain 

http://www.rand.ora/pubs/reprints/2007/RAND_RP1283.pdf 
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2008 
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Accountability 


Arizona State University, 

Education Policy Research Unit, 
2006 
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Accountability 


Arizona State University, 

Education Policy Research Unit, 
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achievement: Problems for the No Child 
Left Behind Act 


Accountability 


Arizona State University, 

Education Policy Research Unit, 
2005 


G. W. Bracey, No Child Left Behind: Where 
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Funding 


Arizona State University, 

Education Policy Studies Lab, 
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General implementation 
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General implementation 
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of schools 
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No Child Left Behind, 2007 


The state of the achievement gap 
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Center on Education Policy, 2009 
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Center on Education Policy, 2008 


C. Scott, A call to restructure 
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Child Left Behind Act in five states 


School improvement & 
restructuring 


Center on Education Policy, 2008 


N. Kober, N. Chudowsky, & V. Chudowsky. 
Flas student achievement increased 
since 2002? State test score trends 
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Achievement 


Center on Education Policy, 2008 
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Center on Education Policy, 2008 


N. Chudowsky & V. Chudowsky, Many 
states have taken a "backloaded" 
approach to NCLB's goal of all students 
scoring "proficient" 


Accountabiiity 


Center on Education Policy, 2008 


Y. Zhang, Some perspectives from rural 
school districts on NCLB 


General implementation 


Center on Education Policy, 2008 


T. W. Fagan, Title 1 funds— Who's gaining, 
who's losing: School year 2008-09 
update 


Funding, costs & 
capacity 


Center on Education Policy, 2007 


N. Kober, N. Chudowsky, & V. Chudowsky, 
Answering the question that matters 
most: Has student achievement 
increased since No Child Left Behind? 


Achievement 


Center on Education Policy and 
Human Resources Research 
Organization, 2007 


L. E. Koger, H. L. Campbell, A. A. Thacker, 
D. E. Becker, & L. L. Wise, Behind the 
numbers: Interviews in 22 states about 
achievement data and NCLB 


Achievement 


Center on Education Policy, 2007 


j . McMurrer, Choices, changes, and 
challenges: Curriculum and instruction 
in the NCLB era 


Curriculum & instruction 


Center on Education Policy, 2007 


A. Minnici & D. D. Hill, Educational 

architects: Do state education agencies 
have the tools necessary to implement 
NCLB? 


Funding, costs & 
capacity 


Center on Education Policy, 2007 


j. McMurrer, Implementing the No Child 
Left Behind teacher requirements 


Teacher quality 


Center on Education Policy, 2007 


C. Scott, Moving beyond identification: 
Assisting schools in improvement 


School improvement & 
restructuring 


Center on Education Policy, 2007 


N. Chudowsky & V. Chudowsky, No Child 
Left Behind at five: A review of changes 
to state accountabiiity plans 


Accountabiiity 


Center on Education Policy, 2007 


C. Scott, Reading First: Locally 
appreciated, nationally troubled 


Reading First 


Center on Education Policy, 2007 


T. W. Fagan, Title 1 funds— Who's gaining, 
who's losing: School year 2007-08 
update 


Funding, costs & 
capacity 


Center on Education Policy, 2007 


A. Minnici &A. P. Bartley, State 
implementation of supplemental 
education services under NCLB 


SES & school choice 


Center on Education Policy, 2006 


D. S. Rentner et al.. From the capital to 
the classroom: Year 4 of the No Child 
Left Behind Act 


Accountabiiity, 
achievement, 
curriculum & 
instruction, funding, 
school improvement, 
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Center on Education Policy, 2006 


C. Scott, Keeping watch on Reading First 


Reading First 


Center on Education Policy, 2006 


T. W. Fagan, A shell game: Federal funds 
to improve schools 


Funding, costs & 
capacity 


Center on Education Policy, 2006 


T. W. Fagan, Title 1 funds— Who's gaining, 
who's losing: School year 2006-07 
update 


Funding, costs & 
capacity 


Center on Education Policy, 2005 


C. Scott & T. Fagan, Ensuring academic 
rigor or inducing rigor mortis? Issues to 
watch in Reading First 


Reading First 


Center on Education Policy, 2005 


D. S. Rentner et al.. From the capital to 
the classroom: Year 3 of the No Child 
Left Behind Act 


Accountabiiity, 
achievement, 
curriculum & 
instruction, funding, 
school improvement, 
SES, teacher quality 


Center on Education Policy, 2005 


N. Chudowsky & V. Chudowsky, States 
test limits of federal AYP flexibility 


Accountabiiity 


Center on Education Policy, 2005 


T. W. Fagan, Title 1 funds— Who's gaining 
and who's losing: School year 2005-06 
update 


Funding, costs & 
capacity 


Civil Rights Project at Harvard 
University, 2008 


G. L. Sunderman (ed.). Holding NCLB 
accountable: Achieving accountabiiity 
and equity 


Accountabiiity 


Civil Rights Project at Harvard 
University, 2006 


G. L. Sunderman & G. Orfield, Massive 
responsibiiities and limited resources: 
The state response to NCLB 


Funding, costs & 
capacity 


Civil Rights Project at Harvard 
University, 2006 


A. Owens & G. Sunderman, School 
accountability under NCLB: Aid or 
obstacle for measuring racial equity? 


Accountabiiity 


Civil Rights Project at Harvard 
University, 2006 


J . Lee, Tracking achievement gaps and 
assessing the impact of NCLB on gaps 


Achievement 


Civil Rights Project at UCLA, 2007 


G. L. Sunderman, Supplemental 
educational services under NCLB: 
Charting implementation 


SES & choice 


Council of Chief State School 
Officers, 2008 


W. J . Erpenbach, Statewide educational 
accountability systems under the NCLB 
Act— A report on 2008 amendments to 
state plans 


Accountabiiity 


Council of the Great City Schools, 
2008 


J. Snipes, A. Horwitz, K. Soga, & M. 

Casserly, Beating the odds. An analysis 
of student performance and 
achievement gaps on state 
assessments: Results from the 2006- 
2007 school year 


Achievement 
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Council of the Great City Schools, 
2006 


M. Casserly, No Child Left Behind in 
America's Great City Schools: Five 
years and counting 


School improvement & 
restructuring, SES & 
choice 


Economic Policy Institute, 2008 


R. Rothstein, R. Jacobsen, & T. Wilder, 
Grading education: Getting 
accountability right 


Accountabiiity 


Economic Policy Institute, 2006 


R. Rothstein, R. Jacobsen, & T. Wilder, 
Proficiency for all— An oxymoron 


Achievement 


Editorial Projects in Education 
(Education Week), 2009 


D. J . Hoff, Schools struggle to meet key 
goal on accountability 


Accountabiiity 


Education Commission of the 
States, 2005 


R. W. DiBiase, State involvement in school 
restructuring under No Child Left 
Behind in the 2004-05 school year 


School improvement 
& restructuring 


Education & the Public Interest 
Center (UC-Boulder) & 
Education Policy Research Unit 
(ASU), 2007 


P. Burch, M. Steinberg, & J. Donovan, 
Supplemental education services under 
NCLB: Emerging evidence and policy 
issues 


SES & choice 


Education Sector, 2007 


M. Yeager, Stiff armed: No Child Left 
Behind's unused funding flexibility 


Funding, costs & 
capacity 


Education Sector, 2006 


K. Carey, Hot air: How states inflate their 
educational progress under NCLB 


Accountabiiity 


Education Trust, 2006 


H. G. Peske, C. Crawford, & B. Pick, 
Missing the mark: An Education Trust 
analysis of teacher-equity plans 


Teacher quality 


Education Trust, 2006 


D. Hall & S. Kennedy, Primary progress, 
secondary challenge: A state- by- state 
look at student achievement patterns 


Achievement 


Education Trust, 2005 


D. Hall, Stalled in secondary: A look at 
student achievement since the No Child 
Left Behind Act 


Achievement 


Government Accountability Office, 
2009 


Access to arts education: Inclusion of 
additional questions in Education's 
planned research would help explain 
why instruction time has decreased for 
some students 


Curriculum & instruction 


Government Accountability Office, 
2008 


No Child Left Behind Act: Education 
actions could improve the targeting of 
school improvement funds to schools 
most in need of assistance 


School improvement 


Government Accountability Office, 
2007 


No Child Left Behind: Education should 
clarify guidance and address potential 
compliance issues for schools in 
corrective action and restructuring 
status 


School improvement & 
restructuring 
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Government Accountabiiity Office, 
2007 


Reading First: States report improvements 
in reading instruction, but additional 
procedures would clarify Education's 
role in ensuring proper implementation 
by states 


Reading First 


Government Accountabiiity Office, 
2006 


No Child Left Behind Act: Education 
actions needed to improve local 
implementation and state evaluation of 
supplemental educational services 


SES & choice 


Government Accountabiiity Office, 
2006 


No Child Left Behind Act: Education 
assistance could help states better 
measure progress of students with 
limited English proficiency 


Accountability 


Government Accountabiiity Office, 
2006 


No Child Left Behind Act: States face 
challenges measuring academic growth 
that Education's initiatives may help 
address 


Accountability 


Government Accountabiiity Office, 
2005 


No Child Left Behind Act: Education could 
do more to help states better define 
graduation rates and improve 
knowledge about intervention strategies 


Accountability 


Government Accountabiiity Office, 
2005 


No Child Left Behind Act: Improved 
accessibility to Education's information 
could help states further implement 
teacher qualification requirements 


Teacher quality 


Great Lakes Center for Education 
Research and Practice, 2005 


E. W. Wiley, W. j . Mathis, & D. R. Garcia, 
The impact of the adequate yearly 
progress requirement of the federal "No 
Child Left Behind Act" on schools in the 
Great Lakes region. 


Accountability 


Human Resources Research 
Organization (HumRRO), 2008 


M.A. Gribbens, H. L. Campbell, &J. 
Mathew, Are advanced students 
advancing? Examining achievement 
trends beyond proficiency. Paper 
presented at AERA annual meeting, 
2008 


Achievement 


Learning Point Associates, 2006 


E. A. Hassel et al.. School restructuring 
under No Child Left Behind: What works 
when? A guide for education leaders 


School improvement & 
restructuring 


Mass Insight Education & 
Research Institute, 2007 


A. Calkins, W. Guenther, G. Belfiore, & D. 
Lash, The turnaround challenge 


School improvement & 
restructuring 


Northwest Evaluation Association, 
2005 


J. Cronin, G. G. Kingsbury, M. S. McCall, & 
B. Bowe, The impact of the No Child 
Left Behind Act on student achievement 
and growth: 2005 edition 


Achievement 
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Northwestern Regional 

Educational Laboratory, 2008 


T. Deussen, K. Nelsestuen & C. Scott, 

Does Reading First work? Data trends in 
five western states. 


Reading First 


Policy Analysis for California 
Education (PACE), UC- 
Berkeley, UC- Davis, Stanford 
Univ., 2007 


B. Fuller, J. Wright, K. Gesecki, & E. Kang, 
Gauging growth: Flow to judge No Child 
Left Behind? Educational Researcher 
36(5), 268-278. 


Achievement 


Policy Analysis for California 
Education (PACE), UC- 
Berkeley, UC- Davis, Stanford 
Univ., 2006 


B. Fuller, K. Gesicki, E. Kang, & J. Wright, 
Is the No Child Left Behind Act working? 
The reliability of how states track 
achievement 


Achievement 


RAND Education, 2007 


L. S. Flamilton et al., Standards-based 
accountability under No Child Left 
Behind: Experiences of teachers and 
administrators in three states 


Accountability 


RAND Education, 2006 


J. A. Marsh & A. Robyn, School and district 
improvement efforts in response to the 
No Child Left Behind Act 


School improvement & 
restructuring 


RAND Education, 2005 


J. S. McCombs, S. N. Kirby, H. Barney, H. 
Darilek, & S. Magee, Achieving state 
and national literacy goals: A long uphill 
road 


Achievement 


Thomas S. Fordham Institute, 
2008 


T. Loveless, S. Farkas, & A. Duffett, Fligh- 
achieving students in the era of NCLB 


Achievement 


Thomas S. Fordham Institute, 
2007 


J . Cronin & M. Dahlin, The accountability 
illusion 


Accountability 


Thomas S. Fordham Institute, 
2007 


J . Cronin, M. Dahlin, D. Adkins, & G. G. 
Kingsbury, The proficiency illusion 


Accountability 


Thomas B. Fordham Institute, 
2006 


C. E. Finn, Jr., L. J ulian, & M. Petrilli, 
2006: The state of state standards 


Accountability 


University of Wisconsin- Madison, 
2007. 


P. Burch, M. Steinberg & J. Donovan, 
Supplemental educational services and 
NCLB: Policy assumptions, market 
practices, emerging issues. Educational 
Evaluation and Policy Analysis, 29(2), 
115-133. 


SES & choice 


The Urban Institute and RAND, 
2008 


J . Flannaway & L. Flamilton, Performance- 
based accountability policies: 
Implications for school and classroom 
practices 


Curriculum & instruction 


U. S. Department of Education, 
Institute of Education Sciences, 
2008 


D. McLaughlin et al.. Comparison between 
NAEP and state mathematics 
assessment results, 2003: Volume 1, 
research and development report 


Achievement 
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U.S. Department of Education, 
Institute of Education Sciences, 
2008 


R. Herman et al.. Turning around 

chronically low-performing schools: A 
practice guide 


School improvement & 
restructuring 


U. S. Department of Education, 
Office of 1 nspector General, 
2008 


The Department's administration of 
selected aspects of the Reading First 
program: Final audit report 


Reading First 


U. S. Department of Education, 
Office of Elementary and 
Secondary Education, 2008 


Findings from the 2007-08 survey on the 
use of funds under Title 1 1 , Part A 


Teacher quality 


U.S. Department of Education, 
Policy and Program Studies 
Service, 2008 


B. C. Gamse et al., Reading First impact 
study: Final report 


Reading First 


U.S. Department of Education, 
Policy and Program Studies 
Service, 2008 


M. Moss et al., Reading First 

implementation evaluation: Final report 


Reading First 


U.S. Department of Education, 
Policy and Program Studies 
Service, 2008 


G. Vernez et al.. State and local 

implementation of the No Child Left 
Behind Act. Volume IV— Title 1 school 
choice and supplemental educational 
services: 1 nterim report 


SES & choice 


U. S. Department of Education, 
National Center for Education 
Statistics, 2007 


Mapping 2005 state proficiency standards 
onto the NAEP scales 


Accountability 


U.S. Department of Education, 
Program and Policy Studies 
Service, 2007 


G. S. Christensen, Evaluation of flexibility 
under No Child Left Behind: Vol. 1 — 
Executive summary of transferability, 
REAP flex, and local flex 


Funding, costs & 
capacity 


U.S. Department of Education, 
Policy and Program Studies 
Service, 2007 


S. Stullich, E. Eisner, & j. McCrary, 
National Assessment of Title 1 final 
report: Vol. 1, implementation 


Accountability, 
achievement, 
curriculum & 
instruction, funding, 
school improvement, 
SES, teacher quality 


U. S. Department of Education, 
Policy and Program Studies 
Service, 2007 


R. Zimmer, et al.. State and local 
implementation of the No Child Left 
Behind Act. Vol. 1: Title 1 school choice, 
supplemental educational services, and 
student achievement 


SES & choice 


U.S. Department of Education, 
Program and Policy Studies 
Service, 2007 


B. Birman et al.. State and local 

implementation of the No Child Left 
Behind Act. Vol. II— Teacher quality 
under NCLB: Interim report 


Teacher quality 


U.S. Department of Education, 
Program and Policy Studies 
Service, 2007 


K. C. Le Floch et al.. State and local 
implementation of the No Child Left 
Behind Act. Vol. 1 1 1— Accountability 
under NCLB: Interim report 


Accountability 
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U.S. Department of Education, 
Office of 1 nspector General, 
2006 


The Reading First program's grant 
application process: Final inspection 
report 


Reading First 


U.S. Department of Education, 
Program and Policy Studies 
Service, 2006 


M. Moss et al., Reading First 

implementation evaluation: Interim 
report 


Reading First 


U.S. Department of Education, 
Program and Policy Studies 
Service, 2005 


E. A. Schenck, D. R. Walker, C. R. Nagel, 
L. C. Webb, Analysis of state K-3 
reading standards and assessments 


Reading First 


Vermont Society for the Study of 
Education, 2007 


Mathis, W. J ., After five years: Revisiting 
the cost of the No Child Left Behind Act. 
In j . K. Rice & C. F. Roellke (Eds.), The 
costs of accountability: Implications of 
high stakes education policies for 
resources and capacities 


Funding, costs & 
capacity 





